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Preface 


Keeping up-to-date with cognitive neuroscience is much 
like surfing the Big Wave at Waikiki Beach. New find¬ 
ings keep rolling in and maintaining a stable balance is 
a big challenge. It is exciting, fun, and, at times, a little 
bit scary. But we keep climbing back on our mental surf¬ 
boards, to catch the coming rollers of advancing science. 
This book aims to provide an overview of the emerging 
science of mind and brain in a way that is accessible to 
students and interested readers at all levels. 

For the Second Edition a number of new features 
have been added. 

1 A new chapter on the Genes and Molecules of 
Cognition (Chapter 16) introduces essential new 
developments in the molecular basis of cogni¬ 
tion. Neurons build links with other neurons by 
expressing proteins, directed by the genetic (and 
epigenetic) apparatus of the cell. Thus the molec¬ 
ular level has become essential for understanding 
learning, language, perception, thinking, and other 
basic functions. This chapter presents 'genes and 
molecules' in readily understandable ways. 

2 A complete revision of the chapter on Consciousness 
and Attention (Chapter 8) takes into account the last 
ten years of research. New recording methods have 
led to remarkable breakthroughs. For example, 
brain rhythms have been shown to carry both con¬ 
scious and unconscious information. The Second 
Edition is fully up-to-date on these findings. 

3 A major revision of Chapter 12, Goals , Executive 
Control , and Action has been included. The frontal 
lobes have been called 'the organ of civilization' 
but they have long been viewed as mysterious. 
Many of the traditional puzzles are now clearing 
up, as this chapter explains, with a presentation of 
the current view of the organizing principles of the 
prefrontal cortex. 


4 Individual scientists are presented in text boxes 
called Frontiers in Cognitive Neurosciences. These 
leading scientists from around the globe present 
their views about new directions and important 
findings in their discipline within the broader field 
of Cognitive Neuroscience: 

• Nelson Cowan, PhD, University of Missouri 

• David Eagleman, PhD, Baylor College of 
Medicine 

• Gerald Edelman, MD, The Neurosciences 
Institute 

• Paul Fletcher, PhD, University of Cambridge 

• Angela Friederici, PhD, Max Planck Institute 
for Human Cognitive and Brain Science 

• Christopher Frith, PhD, Wellcome Trust Centre 
for Neuroimaging 

• Christof Koch, PhD, Division of Biology, 
California Institute of Technology 

• Stephen L. Macknik, PhD, Barrow Neurological 
Institute 

• Susana Martinez-Conde, PhD, Barrow 

Neurological Institute 

• Aniruddh Patel, PhD, The Neurosciences 
Institute 

• Charan Ranganath, PhD, University of 
California, Davis 

• Michael Rugg, PhD. University of California, 
Irvine 

• Jenny Saffran, PhD, University of Wisconsin, 
Madison 

• Larry Squire, MD, University of California, San 
Diego School of Medicine 

5 A new Glossary of technical terms has been added. 
Since vocabulary is one of the great challenges 
in learning cognitive neuroscience, we expect 
the Glossary to be a basic tool for students and 
instructors. 
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PREFACE 


6 A new Mini-Atlas of the Human Brain has been 
added in pullout form at the front of the book. 
Knowing one's way around the brain is the first 
step in understanding the mind-brain sciences. But 
the brain is simply enormous in complexity and 
topographical knottiness. In the First Edition we 
used illustrations from Gray's Anatomy and other 
Elsevier/Academic Press sources to create a lav¬ 
ishly illustrated text. The Second Edition adds a 
Mini-Atlas to give newcomers even more support 
to explore the landscape of the brain - a compass 
and map of the territory. 

7 The Appendix on Brain Imaging Methods has been 
completely updated by an expert in that field. Dr. 
Thomas Ramsoy of the University of Copenhagen. 

As Christopher Frith, Michael Posner, and others 
have written, we are seeing a marriage of the cogni¬ 
tive and brain sciences, building on historic advances 
over the past few decades. Cognitive and perceptual 
mechanisms that were inferred from behavior can now 
be observed more directly in the brain, using a variety 
of novel brain imaging methods. For the first time, we 
can observe the living brain in real time, doing what it 
has evolved to do over hundreds of millions of years. 
The result is astonishingly rich, combining psychology 
and biology, medicine, biochemistry, and physics. Yet 
most scientific studies use well-established psycho¬ 
logical concepts and methods. As a result, we are now 
seeing how psychology and brain science complement 
each other in surprising and gratifying ways. The field 
of cognitive neuroscience is becoming a basic educa¬ 
tional requirement in psychology, biology, education, 
and medicine. 

Cognitive neuroscience has been difficult to cover 
in a single course. Many instructors discover that they 
spend most of the term explaining the brain, with lit¬ 
tle time left for integrative topics. While understand¬ 
ing the brain is vital, an exclusive focus on anatomy 
can defeat the instructors' objectives. 

This text approaches that challenge in several ways. 
First, the body of the text follows the gentlest learning 
curve possible, running along familiar lines: sensory 
perception in vision and audition, working memory, 
attention and consciousness, memory, executive func¬ 
tions, language and imagery, problem solving, emo¬ 
tion, social cognition, and development. The brain is 
introduced step by step, with gradually increasing 
sophistication. To make sense of the material we use a 
functional framework throughout the book. This widely 
accepted set of ideas allows us to see our major topics 


in a single schematic diagram, which grows in depth 
and detail over the course of the book. The functional 
framework can be seen from different perspectives. For 
example, memory stores may be viewed from an active 
working memory perspective; or perception, cogni¬ 
tion, and control may be seen as playing upon per¬ 
manently stored information in the brain (Chapter 2). 
The framework helps either way. 

A website for teachers and students is available at 
http://textbooks.elsevier.com via a free registration. 
Supportive materials for teachers include all figures 
and captions from the book in powerpoint format, 
as well as instructional video and multimedia files. 
Student materials include chapter reviews, quizzes, 
figures, and videos. The support site will be dynamic. 
Materials will be added and changed as warranted by 
new advances, and the authors are happy to consider 
additional ideas and suggestions for new supportive 
materials. 

Instructors may present the chapters in any order 
that suits their goals. For advanced students. Chapters 
4 and 5 on brain imaging and anatomy may be cov¬ 
ered lightly. For introductory courses those chapters 
are essential and may be supplemented with the more 
challenging appendix (by Thomas Ramsoy and col¬ 
leagues). The appendix can also be used as a conven¬ 
ient reference sources. 

A full range of brain disorders are covered, from HM 
and the case of Clive Wearing (Chapters 2 and 9), to 
blindsight, visual neglect, face blindness and other vis¬ 
ual deficits (Chapter 6). Chapter 11 on executive func¬ 
tion covers disorders of undercontrol and overcontrol. 
In certain disorders, motor and cognitive control is not 
directly impaired at all; it seems as if patients are just 
not willing to act. At the other pole, patients sometimes 
spontaneously imitate another person's actions as if 
they cannot stop themselves. Such patients may stand 
up impulsively when the examining physician stands 
up. Disorders of overcontrol and undercontrol reveal 
basic aspects of human executive functioning. 

Some disorders have close psychological analogs. 
Professional musicians, like pianist Van Cliburn, are 
sometimes unable to inhibit their tendency to sing 
along with instrumental playing. Highly trained 
experts can lose some executive control over automatic 
behaviors, especially if they are working under mental 
workload. On the opposite side, a classic symptom of 
severe depression is that patients seem unable to initi¬ 
ate and pursue actions. Brain regions involved in such 
'purely psychological' deficits are often implicated 
in similar organic disorders. We see another striking 
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simplification of the evidence, giving readers a chance 
to understand unifying principles rather than scat¬ 
tered facts. 

Psychological topics are often simplified by brain 
evidence. For example, the verbal part of classical 
working memory - the capacity mentally to rehearse 
numbers and words - is now thought to be a part of 
our normal language capacity. Baddeley (2003) has 
emphasized the discovery that silent rehearsal acti¬ 
vates the well-known speech regions of cortex. Thus 
the 'phonological loop' of traditional working memory 
is no longer seen as a separate cognitive feature, but 
rather as a silent way of using speech cortex. Similarly, 
Kosslyn and others have shown that visual imagery 
makes use of a subset of the cortical areas involved in 
normal visual perception (2004). Even more surpris¬ 
ing, visual attention appears to be closely related to 
eye movement control. Athletes and musicians use the 
sensorimotor brain to engage in silent mental practice. 
Thus 'inner' and 'outer' processes seem to involve 
overlapping regions of the brain, a major simplifica¬ 
tion of the evidence. 

While cognitive neuroscience does not always sim¬ 
plify things, it does so often enough to allow us to 
organize this text along recurring themes. This makes 
the material easier to teach and understand. It allows 
us to explore a wide range of basic topics, including 
emotion, social cognition, and development. 

The companion materials are designed to enrich stu¬ 
dent learning by way of vivid classroom demonstra¬ 
tions, images, and learning points, using Powerpoint 
presentations and movie clips. A number of phenom¬ 
ena in the text can be best illustrated by way of experi¬ 
ments and movie clips. For example, a patient is shown 
with locked-in syndrome, able to communicate only 
by means of eye movements directed at a keyboard 
display. For comparison, we show patients who look 
superficially the same, but who are suffering from true 
coma. 

At the end of each chapter, review questions and 
brain drawing exercises are designed to help students 
learn interactively. We particularly emphasize draw¬ 
ing and coloring exercises as a way to grasp the knotty 
three-dimensional organization of the brain. 

This text covers some frontier topics in the ever- 
changing vista of cognitive neuroscience. One popular 
topic is the relationship of 'the mind' as we experi¬ 
ence it and 'the brain' as we observe it: i.e. the historic 
topic of consciousness and its brain correlates. Alan 
Baddeley recently noted that, 'Perhaps the greatest 
change over the last twenty years within cognitive 


psychology and cognitive science . . . has been the 
acceptance of consciousness as a legitimate and tracta¬ 
ble scientific problem'. 

The renewed acceptance of consciousness has 
changed research in perception, memory, and atten¬ 
tion, as seen in pioneering work by Endel Tulving, 
Daniel Shachter, Gerald Edelman, Francis Crick, 
Christof Koch, and numerous others. While some 
textbooks have added chapters on consciousness, we 
believe that the topic has now become so pervasive 
that it needs to be addressed throughout. As Science 
journal recently noted in its 125th anniversary issue, 
consciousness is now thought to be one of the major 
unsolved problems in biological science. While much 
remains to be learned, psychologists have long studied 
conscious processes under such headings as 'explicit 
cognition' and 'focal attention'. Those constructs are 
all assessed by the behavioral index of accurate report, 
which has been taken to signal conscious events since 
the beginning of psychophysics, some two centuries 
ago. Thus 'consciousness' can be seen as an umbrella 
label, much like 'memory' and 'perception', with a 
number of subtopics like subliminal perception, auto¬ 
biographical memory, and focal attention. 

Voluntary control is also back on the forefront of 
research, sometimes under the rubric of 'strategic 
control' and 'executive functions'. In the brain, volun¬ 
tary and non-voluntary functions can be clearly dis¬ 
tinguished anatomically and physiologically. Robust 
differences also appear in functional brain imaging 
and behavior. Finally, the notion of executive control 
appears to be moving toward new insights on the 
'self' of everyday life, as studied in social and person¬ 
ality psychology. 

All these topics show a striking convergence of 
behavioral and brain evidence. 

The brain basis of emotion and social relationships 
is developing as well. 'Mirror neurons' are involved 
with the ability to perceive intentions in others; 
unconscious 'threat faces' can stimulate the amygdala; 
and conflicting aspects of self-control are apparently 
played out in competing impulses in prefrontal cortex. 

Cognitive neuroscience is challenging; it is also one 
of the most important frontiers in science. Students 
will be rewarded with a new depth of understand¬ 
ing of human nature, one that has never been quite as 
clear and convincing as it is today. 

The editors are especially grateful to Dr. Johannes 
Menzel, Publisher, Science Solutions and Content 
Strategy, Elsevier Publishers. We are also very grateful 
to Clare Caruana, Development Editor, Life Science 
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Books, Academic Press, Elsevier, for her guidance 
and support throughout our efforts for this Second 
Edition. During the preparation for our Second Edition, 
Johannes and Clare moved on to new and greater chal¬ 
lenges at Elsevier and we welcomed Mica Haley, Senior 
Acquisitions Editor, and Melissa Turner, Development 
Editor, to our team. Mica and Melissa picked up the 
Second Edition and seamlessly guided us through the 
complex stages of producing a large text. We are grate¬ 
ful for all of their efforts on our behalf. We appreciate 
their consistent kindness, advice, and support through¬ 
out a very challenging project. Indeed, the current text 
would have been impossible without the vast archival 
resources of Elsevier. 

We are grateful for our many contributors who 
added their expertise to this volume: 

Daniela Balslev 

Danish Research Centre for Magnetic Resonance, 
Hvidovre Hospital, Denmark 

Dmitri Bougakov 

Department of Neurology, New York University 
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Katharine McGovern 

California Institute for Integrative Studies, 
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Morris Moscovitch 
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Canada 

Olaf Paulson 
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Toronto, Ontario, Canada 

Frank Tong 

Department of Psychology, Vanderbilt University, 
Nashville, Tennessee, USA 

The first editor is particularly grateful to Dr. Gerald 
Edelman and his colleagues at the Neurosciences 
Institute in San Diego, as a unique source of insight 
and collegial guidance on many topics in neuroscience. 
He is also indebted to the Mind Science Foundation of 
San Antonio, Texas, for supporting pioneering explo¬ 
rations in the cognitive neuroscience of consciousness. 
MSF's Board and Executive Director, Joseph Dial, have 
been especially active in working to unify behavioral 
and brain approaches to conscious experience (www. 
mindscience. org). 

A number of colleagues and friends have helped 
us to gain a deeper understanding of mind and brain. 
There are too many to list here, but we want them 
to know of our gratitude. Among recent colleagues, 
Stan Franklin, Walter Freeman, William P. Banks, E. 
R. John, Christof Koch, Francis Crick, Karl Pribram, 
Dan Dennett, Patricia Churchland, Patrick Wilken, 
Geraint Rees, Chris Frith, Stan Dehaene, Bjorn Merker, 
Jaak Panksepp, Stu Hameroff, Thomas Ramsoy, Antti 
Revonsuo, Michael Pare, Henry Montandon, Murray 
Shanahan, and many others have helped us to think 
through the issues. We have not always agreed, but 
we have always felt grateful for their constantly inter¬ 
esting and important ideas. 

We have been fortunate to have the help of out¬ 
standing co-authors for this book. Their contributions 
are acknowledged in the Contents. This book would 
be poorer without their depth of knowledge and 
desire to communicate. 

In a broader but very real sense, this book owes 
everything to the community of scholars and scientists 
who are cited in its pages. 

In more personal terms, the first editor would like 
to acknowledge his loved ones for putting up with 
him during this daunting project. His parents, now 
deceased, are still a constant source of inspiration and 
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guidance. He would also like to express his gratitude 
to Dr. Barbara Colton for constant feedback and sup¬ 
port, which have made the book much more readable 
and the editor far more sensible. 

Ms. Blair Davis was invaluable in helping to put 
this book through its final stages. Mr. Shawn Fu pro¬ 
vided a number of beautiful brain illustrations. 

The second editor is particularly grateful to the 
first editor for sharing his concept and passion for 
this project. Our collaboration on this text began a 
few years ago during a 3-hour layover in Chicago and 
has blossomed into this full-blown project, along with 
a strong friendship. She is also indebted to many 


colleagues and friends with whom she has learned 
about how language gets wired up in the brain. There 
are too many to include here, but she is indebted to 
them all. She is particularly grateful to Greg Hickok, 
David Poeppel, Larry Cahill, Norm Weinberger, Bryna 
Siegal, Anne Spence, and Kourosh Saberi for good 
conversations, ongoing debates, and thoughtful dis¬ 
cussions through the years. 

The second editor would like to acknowledge 
her family for their constant support during this 
project. Last, she is indebted to Kim for his insight 
and love. 
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COGNITION, BRAIN, AND CONSCIOUSNESS 


SECOND EDITION 



. . . from the brain, and from the brain alone, arise our pleasures, joys, laughter and jokes, as 
well as our sorrows, pains, griefs and tears. Through it, in particular, we think, see, hear, and 
distinguish the ugly from the beautiful, the bad from the good, the pleasant from the unpleasant 
... all the time the brain is quiet, a man can think properly. 

Attributed to Hippocrates, 5th century BCE 
(quoted by Kandel etal., 1991). 
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Upper left: The human body and its basic orientation planes. Lower left: A standard view of the brain from the left side. 
The left hemisphere is 'looking' left. The light blue region in the back of the brain is the occipital lobe. The diagram 
on the lower right shows a 'neural hierarchy', a simplified way of showing neural connections in the cortex, and on 
the upper right we see the entire cortex as a 'circle of hierarchies'. The yellow arrow in the center depicts a common 
view of the role of perceptual consciousness in the brain. Source: Drake et al., 2005. 
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1.0 INTRODUCTION 

This chapter gives an overview of cognitive neuro¬ 
science, the combined study of mind and brain. The 
brain is said to be the most complex structure in the 
known universe - with tens of billions of neurons, 
connected by trillions of transmission points. It can be 
changed by taking in molecules, as in drinking a glass 
of wine, and by external stimulation, like listening to 
exciting news. Some neuronal events happen over a 
thousandth of a second, while others take decades. 
In spite of this vast range of working conditions, we 
know many facts about the mind-brain that are basic 


and fairly simple. This book aims to let those facts 
stand out. 


2.0 AN INVITATION TO 
MIND-BRAIN SCIENCE 

It is hard to talk about the last dozen years in cognitive 
neuroscience without using words like 'remarkable' 
and 'revolutionary'. In a sense, a century of behav¬ 
ioral and brain science has received resounding con¬ 
firmation with the new technology of brain imaging, 
the ability to observe the living brain in real time. That 
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FIGURE 1.1 Rembrandt, The Anatomy 
Lesson of Dr. Tulp. This historic painting by 
Rembrandt shows the excitement of the 
first revolution in scientific thinking about 
the human brain and body. Dr. Tulp, on the 
right, is demonstrating how the muscles of 
the forearm control hand movements. The 
systematic dissection of human cadavers 
signaled a rebirth of careful empirical obser¬ 
vation which still guides us today. Source : 
Masquelet, 2005. 


does not mean, of course, that we have merely con¬ 
firmed what we thought we knew. Rather, the ability 
to record from the living brain has proved to be fruit¬ 
ful in bringing out new evidence, raising new ideas 
and stirring new questions. Many scientists have a 
sense that a great barrier - between the study of mind 
and brain - is being bridged. Historically tangled 
questions continue to yield quite beautiful insights. 

Along with this feeling of progress has come a great 
expansion. Just 10 years ago, behavioral scientists 
might not have seen a connection between human 
cognition and the genetic revolution, with brain mol¬ 
ecules, or with the mathematics of networks. Today, 
those topics are all part of a connected island chain 
of knowledge. Previously avoided topics are now 
anchored in plausible brain correlates - topics like 
conscious experience, unconscious processes, mental 
imagery, voluntary control, intuitions, emotions, and 
even the self. Some puzzles seem bigger than before - 
the nature of the permanent memory trace, for example. 
There seem to be more continuities than ever before 
between psychological and brain studies of percep¬ 
tion, memory, and language. 

In some cases, brain evidence helps to resolve puzzles 
that psychologists have wrestled with for decades. For 
example, in the study of attention a debate has raged 
between 'early' and Tate selection' of attended informa¬ 
tion. People may pay attention to a coffee cup based on 
low-level visual features like color, texture and location. 
Alternatively, they might focus on the coffee cup based 


on higher-level properties like 'usefulness for drink¬ 
ing hot liquids'. Evidence can be found for both. After 
decades of debate, brain studies have now shown that 
attentional selection can affect neurons at almost any 
level of the visual system. The answer therefore seems 
to be that there can be both early and late selection, as 
many psychologists have also argued. In many cases 
like this we find surprising convergence between brain 
and behavioral evidence. 

3*0 SOME STARTING POINTS 

3*1 Distance: seven orders of magnitude 

To understand the mind-brain, it helps to have an idea 
of its orders of magnitude, the powers of ten that tell 
us the basic units of interest. From front to back, a 
brain is perhaps a seventh of a meter long. If you take 
one step, the length of your stride is about one meter 
(a little more than a yard). If you raise that length to 
the next order of magnitude, 10 meters, you might 
obtain the rough length of a classroom. One hundred 
meters is a standard sprinting distance, and a thou¬ 
sand meters or one kilometer is a reasonable length 
for a city street. By the time we raise the exponent to 
10 7 meters, or 10 000 km, we are already at 6000 miles, 
the distance from coast to coast in North America, or 
from Paris to the equator in Europe and Africa. That 
is ten million steps. In order to understand the most 
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TABLE 1.1 Distance from 10 7 m to one meter 


1 If you drink alcohol your brain will change. Alcohol molecules are about 10 -7 meters in size (Figure 1.2) (0.0000001). 

2 If you swallow a tranquillizer, the chemical concentration that flows across some of your synapses will be changed, at 10 -6 m for the width of a 
synapse (one-millionth of a meter, or one micron). 

3 The cell bodies of neurons are about a hundred times larger than synapses, around 10 -4 . 

4 However, neurons do not act alone; they are grouped with others in small networks, perhaps ten times larger than single neurons. Small cortical 
columns are about 10 -3 m or one millimeter in diameter. We have now reached one-thousandth of a meter, which we can see with the naked eye. 

5 Brain maps, such as those in the visual system, can be quite large. The first cortical map of the light array is called area VI, the first visual region of 
cortex. It is about the size of a credit card, several square centimeters in size, or 10 _2 m 2 . 

6 Visual maps don't work in isolation. They are part of brain-wide activation networks that interact in constantly changing patterns. Such 
widespread networks operate at an order of magnitude of ten centimeters (10 _1 ), well inside the visible range if you could see brain activity 
through the scalp. Indeed, some current methods observe the brain's constantly varying activity by simply shining a laser through the shaved 
scalp of rats or other small mammals. One can watch the flow of blood to brain surfaces that are active, just as we can see increased blood flow to 
the skin when people have been running hard. 



FIGURE 1.2 Spatial powers of ten. (a) A brain image of a subject looking at a rotating black and white stimulus, 
so that his visual cortex (in the rear of the brain) is intensely stimulated, (b) A midline view of the cortex, with area 
VI marked - the first place where the visual pathway reaches cortex. VI is about the size of a credit card, (c) The 
head of a fruit fly. The fruit fly brain has about 1 000 000 neurons. A single neuron is shown in (d). Neurons vary in 
size, but they are extraordinarily small; we have tens of billions in our brains, (e) A dopamine molecule. Dopamine 
plays an essential role in working memory, in the experience of pleasure, and in the control of muscles. Parkinson's 
disease is one result of a decline of the dopamine system. From (a) to (e), the range of sizes involves about seven 
orders of magnitude. 


important magnitudes of the brain we can simply 
imagine going the other way, seven orders of magni¬ 
tude from one meter to 10“ 7 (Table 1.1). Considered 
this way it is an awesome prospect in size and 
complexity 


Visible behavior takes place anywhere from a cen¬ 
timeter and up. A finger striking a keyboard moves 
only a few centimeters. When we speak, our tongue 
moves only a centimeter or two. A single walking step 
is about a meter long. Most people are a little less than 
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FIGURE 1.3 Small molecules can 
change the brain. Some of the smallest mol¬ 
ecules like nitrous oxide (N 2 0) can change 
specific brain functions. That came as a big 
surprise to Western medical doctors around 
1800, like the ones above, shown in a draw¬ 
ing by a medical student in 1808. However, 
such facts continue to surprise us. Nitric 
oxide (NO), which is toxic when breathed, 
is produced in tiny quantities as a crucial 
neurotransmitter. The erectile drug, Viagra, 
promotes NO transmission in penile blood 
vessels. Source : Adelman and Smith, 2004. 


two meters in height, and the longest neurons in the 
human body may be about 1 meter. 

3.1.1 A note about neurochemicals: 
the smallest level 

Neurotransmitters range in size, and diffuse across gaps 
between neurons - the synapses - which vary between 
25 nanometers to 100 micrometers (Iversen, 2004). Most 
brain-changing chemicals promote or block molecular 
communication between nerve cells. The list of every¬ 
day chemicals that change the brain includes nicotine, 
alcohol, oxygen (in the air), toxic gases like carbon mon¬ 
oxide, glucose from the liver and sucrose from foods, 
chocolate, coffee, nerve toxins like lead, and a long list 
of medications (Figure 1.3). It is hard to overstate the 
importance of such molecules in everyday life. 

Molecular messengers in the brain can be divided 
into two large groups. The first group, the neuromod¬ 
ulators, are 'sprayed' throughout large parts of the 
forebrain from small fountain-like clumps of cell bod¬ 
ies at the base of the brain. These are informally called 
'spritzers', because they secrete neurochemicals from 
widely dispersed axons, to modulate large parts of 
the brain. However, neuromodulators can have local 
effects if they lock into specific types of local receptors. 
For example, while dopamine is spread very widely, 
the D1/D2 dopamine receptors are believed to have 
local effects in the frontal cortex related to working 


memory (Gibbs and D'Esposito, 2006). Thus, a very 
wide-spread neuro-modulator, dopamine, can have 
more local effects when it locks into receptors in a spe¬ 
cific region of the brain. 

The second major group of neurotransmitters have 
much more localized actions. They are mostly peptides, 
i.e. small subunits of proteins, which are secreted directly 
into synaptic gaps. More than 40 types of neuropeptides 
have now been found throughout the brain. The two 
best-known examples are glutamate, the most wide¬ 
spread excitatory neurotransmitter in the cortex, and 
GABA , the most common inhibitory neurotransmitter. 

Scientific advances often follow our ability to 
observe at different magnitudes. The wave of discov¬ 
eries we are seeing today results from our new ability 
to observe the living brain. The ability to observe over 
some seven orders of spatial magnitude makes mind- 
brain science possible. 


3*2 Time: ten orders of magnitude 

Human beings function over a great range of time scales 
(Table 1.2). Behaviorally, one-tenth of a second (100 ms) 
is an important unit to keep in mind. The fastest (simple) 
reaction time to a stimulus is about 100 milliseconds, 
and the time it takes for a sensory stimulus to become 
conscious is typically a few hundred milliseconds. This 
makes sense in the environment in which human beings 
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TABLE 1.2 Time: Ten orders of magnitude, from years to milliseconds 


1 When you listen to a high musical note, your auditory nerve is firing as fast as the sound waves that vibrate your ear drum, up to 1000 
times per second, or one wave per millisecond (ms). (Reminder: cycles per second = Hertz = Hz). 

2 Neurons can fire as fast as 1000 Hz, although the average neuron in the cortex fires about ten times per second. 

3 The auditory nerve is a bundle of nerve axons, each of which fires slower than 1000 cycles per second, but together they can respond 
at millisecond speed. Some sensory events are even faster. In a remarkable feat, our brain can distinguish differences between sounds 
arriving at the two ears, down to several microseconds , or millionths of a second. That is 1000 times faster than the fastest firing rate of a 
neuron (see Chapter 7). However, for our purposes, milliseconds are the most useful unit at the small end of the time scale. 

4 By comparison, if you count to yourself from one to ten, your counting takes place on the order of seconds, or tens of seconds. If you 
study this chapter carefully, it will take you an hour or more - more than three thousand seconds. (Sorry about that!) 

5 Going back down the scale again, if the sound track of a movie is delayed by a fraction of a second behind the video, you will notice a 
disconnect between the sounds and the movements of an actor's mouth. The break-up of audio-visual synchrony occurs after about 
100 milliseconds (ms) lag, or one-tenth of a second. This is another reason to think of one-tenth of a second as a special order of 
magnitude. 

6 One hundred milliseconds is also about the fastest we can react to an event that we know will come soon, as in starting a car at a 
traffic light (simple reaction time). One-tenth of a second is also about the rate of individual alpha and theta waves in the brain. (Alpha 
waves range between 8-12 Hz, and theta between 5-7Hz.) It is also the order of magnitude required to become conscious of a sensory 
stimulus. 


evolved. If you took several seconds to react to a hungry 
predator, you will soon provide it with a tasty protein 
snack. Biologically, you would not get a chance to repro¬ 
duce. On the other hand, if you tried to react as fast as 10 
milliseconds - one-hundredth of a second - you would 
be driving your brain faster than it could combine the 
sights and sounds of a running tiger. It would be hard to 
tell what direction a predator might be coming from. So 
the 100 ms range gives a useful middle ground. 

Brain events at different time and spatial scales go 
on at the same time, like the elements of a symphony - 
notes, phrases, and whole musical movements. When 
you listen to a song, you are conscious of only a few 
notes at any time, but the notes you hear fit into a 
larger cognitive structure which makes it possible to 
appreciate how the entire song is developing. Movie 
frames are shown at 24 images per second, or about 
40 milliseconds per frame, to show smooth move¬ 
ment. (That's why they call them movies!) Slower 
rates than 24 Hz start to look jerky, like the early silent 
movies. However, the plot of a movie takes place over 
minutes and hours. In a mystery film, if you cannot 
remember the crime at the beginning, the fact that the 
perpetrator is found at the end will not make sense. 
Thus, understanding movie plots requires cognitive 
integration over many minutes. All these time scales 
must somehow be combined. Each kind has a struc¬ 
ture and a typical time range. The brain keeps track of 
all simultaneously. 

At the longer end of the time scale, it can take years 
to learn a difficult skill, like skiing or playing guitar. 
Infants learn their first language over several years, 
while adults tend to keep their basic personality struc¬ 
ture over decades. Such long-term processes depend 


upon the same brain as 100-millisecond events. In the 
time domain, therefore, we need to understand about 
ten orders of magnitude, from one-thousandth of a 
second (a millisecond) for a single neuron to fire, to 
more than 100 000 seconds per day, and tens of mil¬ 
lions of seconds per year. 

3.3 The need to make inferences - going 
beyond the raw observations 

Science depends on a constant process of inference, 
going from raw observations to explanatory concepts. 
Thousands of years ago, when human beings began to 
wonder about lights in the sky like the sun, the moon, 
and the stars, they noticed that some were predictable 
and some were not. The 'wanderers' in the night sky 
were called planete by the Greeks, and we call them 
'planets' in English. These wandering lights became a 
source of fascination. It was not until the 17th century 
that their paths were understood and predicted. The 
solution to the wandering lights puzzle was to realize 
that the planets were giant earth-like spheres revolving 
in orbit around the biggest object of them all, the sun. 
It took centuries of argument and observation to settle 
on that solution. Isaac Newton had to invent the infini¬ 
tesimal calculus to bring the debate down to a simple 
equation: planetary orbits can be predicted from the 
simple fact that gravitational force equals the mass of 
the orbiting planet times its acceleration. Notice that all 
those words - 'sun', 'planet', 'force', and 'gravity' - are 
inferred concepts. They are far removed from the first 
observations of lights in the sky (Figure 1.4), yet they 
explain those raw observations: they are explanatory 
inferences. 
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Observer 


FIGURE 1.4 Making inferences about lights 
in the night sky. To an earthbound observer the 
planets look like wandering lights in the night sky. 
After many years of careful astronomical observa¬ 
tions, Isaac Newton and others determined that 
the complex wandering path of the planets reflects 
a very simple reality. The leap from raw observa¬ 
tion to inferred concepts and explanations is a cru¬ 
cial part of science. 


All science depends upon careful observations and 
conceptual inferences. The resulting explanatory frame¬ 
work has been called a 'nomological network' - that is, 
a network of labeled concepts and relationships, which 
together provide us a sense of understanding. Along 
the way, successful science allows us to make more 
accurate predictions, and to apply the resulting knowl¬ 
edge in ways that sometimes transform life. It all begins 
with exact observations and plausible inferences. 

These basic ideas have a direct bearing on cogni¬ 
tive neuroscience. When we talk about cognition - 
language, learning, or vision - we also use inferred 
concepts , which must be firmly anchored in reliable 
observations. For example, the capacity of immediate 
memory - the kind we can mentally rehearse - is about 
seven plus or minus two items, as George A. Miller 
famously noted in a paper called 'The magical number 
seven plus or minus two' (1956). That number seems 
to apply to many kinds of randomly selected items: 
colors, numbers, short words, musical notes, steps on 
a rating scale, and so on. The recent consensus is that 
the actual capacity of immediate memory is even less 
than seven, about four different items (Cowan, 2001). 
But the most important point is the remarkable consist¬ 
ency in the data. Try to remember ten different foods 
on your shopping list, for example, and you will find 
that only about seven are remembered - and if you are 


busy thinking about other things, that number drops to 
four. It is an amazingly narrow limit for a giant brain. 

There are only a few basic conditions for obtain¬ 
ing the size of working memory. One is that each item 
must be attended for only a brief time - perhaps sev¬ 
eral seconds - so that it cannot be memorized well 
enough to enter permanent memory. A second con¬ 
dition is that the items must be unpredictable from 
existing knowledge. If we ask people to remember a 
regular number series like 0, 5, 10, 15, 20, 25 . . . they 
only need to remember the rule, and it will seem that 
their working memory capacity is endless. Cognitive 
concepts like 'working memory' are the product of 
decades of experimental observations which finally 
become so solid that we can summarize the evidence 
in one basic concept (Figure 1.5). 

Ideas like working memory have turned out to be 
useful, but it is quite possible that we will find a more 
attractive way to think about them tomorrow. All 
inferred concepts are somewhat tentative. Newton's 
idea of gravity dominated physics for three centuries, 
then Einstein found another way to look at the evi¬ 
dence. Scientific concepts are not metaphysical cer¬ 
tainties. They are always subject to revision. 

Cognitive neuroscience is also based on inferences 
from raw observations. Because brain scans have the 
appearance of physical objects that we can see and 
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FIGURE 1.5 Cognitive concepts 
are based on consistent behavioral 
observations. Concepts like 'working 
memory' are not given in nature. They 
emerge after many years of testing, 
when a large body of evidence seems 
to be explained by an inferred con¬ 
cept. Working memory was proposed 
in 1974 after two decades of study 
of immediate memory. Today it has 
expanded in scope, so that visual, ver¬ 
bal, and other temporary buffers are 
called working memories. 


BOX 1.1 Imaging the living brain 


The very idea of observing the living brain in action was 
unimaginable a decade or two ago. Methods from phys¬ 
ics and molecular biology have been applied to the for¬ 
midable problem of recording brain activity. A perfect 
method would be able to follow tens of billions of neu¬ 
rons, and sample each one a thousand times per second. 
It should then be possible to track the constantly shifting 
interplay between smaller and larger groups of neurons, 
involving trillions of possible connections. By analogy, 
a perfect spy satellite in orbit around the earth would 
be able to see every single human being, as well as the 
changing relationships between individuals and groups, 
from families to entire nations. 

Such a perfect method does not exist. Our understand¬ 
ing of the brain is a kind of collage of many fragments of 
the puzzle, glued together to make a reasonable picture 
of the whole. But thinking about a perfect observation 
method gives us the parameters we might aim for. 

Brain imaging has been a breakthrough technology for 
cognitive neuroscience, adding new evidence to decades 
of cognitive psychology, behavioral conditioning meth¬ 
ods, psychophysics and fundamental brain science. Before 
these techniques matured, our knowledge came from ani¬ 
mal studies and from the haphazard injuries incurred by 


human beings. But brain injuries are extremely imprecise, 
and even to describe the damage, neurologists often had 
to rely on post-mortem examination of patients' brains. 
The brain can often compensate for injuries, and lesions 
change over time as cells die and adaptation occurs, so 
that post-mortems do not necessarily reflect the injury at 
the time of diagnosis. Animal studies relied on presumed 
homologies - i.e. similarities across species - which were 
often not persuasive to everybody. No other animals 
besides humans have language and other distinctive 
human specializations. It was therefore very difficult to 
understand brain functions. 

Many of those problems were resolved when it became 
possible to observe the living human brain, first by elec¬ 
troencephalography (EEG), then by X rays, then computer 
tomography (the study of slices - from the Greek word 
for 'slice', tomos - based on X-rays). Today, we have more 
than a dozen techniques that are rapidly evolving toward 
greater precision and a broader range of application. The 
most widely used methods are EEG, positron emission 
tomography (PET), magnetic resonance imaging (MRI), 
functional MRI, and magnetoencephalography (MEG). See 
Chapter 4 for a detailed description of these and other new 
techniques for investigating the dynamic human brain. 


touch, we are tempted to think that we are seeing 
haw reality' in brain scans. But that is a seductive fal¬ 
lacy. Electroencephalography (EEG) is an inferential 
measurement of brain activity, as is functional mag¬ 
netic resonance imaging (fMRI), positron emission 
tomography (PET), and all the other advanced tools 


we have today (Box 1.1). Even recording from neurons 
only gives us a tiny sample of single cell firing among 
tens of billions of cells. Neurons make perhaps ten 
thousand connections, and there is evidence that even 
the input branches of a single neuron (the dendrites) 
may compute information (Alle and Geiger, 2006). 
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FIGURE 1.6 Brain measures of 
working memory are also inferential. 
Working memory functions in the 
brain have been studied using behav¬ 
ioral measures, but also with fMRI, 
EEG, and single-neuron record¬ 
ings. Each of these measures has 
its pros and cons, but none of them 
is the 'ultimate measure' of work¬ 
ing memory. Overall, brain indices 
of working memory converge well 
with behavioral measures. Cognitive 
neuroscience is based on the study of 
such combined sources of evidence, 
but we must be prepared to find that 
our current concepts may be inter¬ 
preted in a different way. 


Therefore, measuring the electrical activity of single 
neurons is only a tiny sample of a very complex dance 
of molecules and electromagnetic fluxes. Recent imag¬ 
ing techniques are extraordinarily useful, but they still 
involve inferences about the working brain. 

Yet we must make some simplifying assumptions - 
that is how science develops. It is just important to 
keep in mind what the assumptions are, and to be pre¬ 
pared to change them if necessary. Figure 1.6 illustrates 
this point. In cognitive neuroscience we make infer¬ 
ences based on behavioral and brain observations. We 
don't observe 'attention' or 'working memory' directly. 
For that reason, it is essential to understand the nature 
of the evidence that we use to make those inferences. 

3*4 The importance of convergent measures 

When that mythical first cave dweller pointed to a star 
at night, we can imagine that nobody else in the clan 
believed him or her. What lights in the sky? The sky 
was the abode of the gods; everybody knew that. That 
kind of skepticism is the norm. When Galileo first used 
a crude telescope to look at the moons of Jupiter, some 
critics refused to look through the telescope, since 
they held that only the naked eye could tell the truth. 
Skepticism is still the norm, and science always makes 
use of converging measures to verify observations. 
Today, any major hypothesis in cognitive neuroscience 
is tested over and over again, using single-neuron 


recordings, animal studies, EEG, fMRI, MEG, and 
behavioral measures such as verbal reports and reac¬ 
tion time. No single study settles a hypothesis. Every 
major claim requires multiple sources of support. 

Part of the debate is focused on exactly what it is 
that is being measured. Every new method of observa¬ 
tion is met with that kind of question. The most popu¬ 
lar method today is functional magnetic resonance 
imaging (fMRI). But as we will see, there is ongoing 
debate about what it is that fMRI measures. The same 
is true of behavioral measures of working memory, 
single cell recordings, EEG, and all the rest. 

3*5 Major landmarks of the brain 

How does brain activity relate to cognition? We will 
present functional brain images to guide you in inter¬ 
preting the studies presented in this text. But brain 
function is always grounded in anatomy, and we will 
cover the basic geography of the brain for that reason. 
Figure 1.7 (top) shows the outside view of the left hem¬ 
isphere, also called the lateral view. Below it is the medial 
view of the right hemisphere, also called the mid sagit¬ 
tal section of the brain. It is a slice through the midline, 
from the nose to the back of the head. Every other slice 
that runs parallel to it is called sagittal (see Figure 1.8). 

It is important to learn the major landmarks in the 
brain. Some of the most important ones are the big 
folds or valleys in the cortex, the outer structure of 
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the brain. The largest valley runs along the midline 
between the right and left hemispheres and is called 
the longitudinal fissure. A second large fold runs for¬ 
ward at a slant along the side of the brain, and is called 
the lateral sulcus (from the word for the 'ditch' or 'fur¬ 
row'). The lateral sulcus divides the 'arm' of the tem¬ 
poral lobe from the 'body' of the main cortex. Since the 
temporal lobe always 'points' in the direction of the 
eyes, identifying it is the easiest way to tell which way 
the brain is facing. Spotting the temporal lobe is one of 
the first things to do when looking at a brain picture. 

The corpus callosum , another major landmark, is 
a great fiber bridge flowing between the right and 
left hemispheres. It is visible on the upper portion of 
Figure 1.7 as a curved section that begins behind the 
frontal lobe and loops up and to the back, ending 
just in front of the cerebellum. When the corpus cal¬ 
losum is cut, it looks white to the naked eye because 


FIGURE 1.7 The brain: medial and lateral 
views. The top panel shows a medial view of the 
right hemisphere with major structures high¬ 
lighted. This view is also called the mid sagittal 
section of the brain. The lower panel shows a view 
of the left hemisphere from a lateral (outside) view¬ 
point. The front of the brain is on the left side of 
the figure and the back of the brain is on the 
right side. The four lobes and the cerebellum are 
labeled. Source : Drake et al, 2005. 

it consists of white matter (i.e. nerve axons covered 
by white myelin cells, filled with fat-like lipid mole¬ 
cules). The corpus callosum was called the 'calloused 
(or tough) body', because that is how it appeared to 
early anatomists who named these structures. It was 
discovered early on, because it can be exposed simply 
by gently separating the two great hemispheres. In 
Figure 1.9, the corpus callosum is shown beautifully 
in a classic drawing by the great Renaissance painter 
Titian, drawn for the first detailed book of anatomy by 
Andreas Vesalius. 

A final landmark is the central sulcus, which divides 
the rear half of the brain (the posterior half) from the 
frontal lobe. The posterior cortex is predominantly 
sensory, with visual, spatial, auditory and body-sense 
regions, while the frontal lobe is motor and cognitive. 
The central sulcus is a clear dividing line between the 
input-and output-related areas of cortex. 
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FIGURE 1.8 The major planes of section (cuts). The three 
main slices or sections of the brain. Top panel shows a vertical sec¬ 
tion of the brain, called sagittal, from the front of the brain to the 
back. When the slice is exactly through the midline, between the 
two hemispheres, it is called mid sagittal. The center panel shows a 
horizontal slice through the brain. The lower panel shows a coronal 
section (named for its crown shape) like a sliced sausage. (These 
terms have synonyms that are explained in Chapter 5.) 

Locating these three major folds is the first step in 
orienting yourself. 

The cortical lobes flow over to the inside of each 
half cortex, as we will see. Because it is not easy to 



FIGURE 1.9 Top : Andreas Vesalius, showing a dissected arm 
and hand. Andreas Vesalius was a Belgian physician (1514-1564) 
who overturned the traditional teaching of anatomy by performing 
post-mortem dissections of human bodies. He is shown here display¬ 
ing the exposed hand and arm. The arm and hand were important 
to Vesalius as evidence for the divine hand in worldly affairs. Until 
Vesalius, it was widely believed that women had one less rib than 
men, based on the Biblical story of Adam and Eve. Real dissections 
of human bodies were not performed, and accurate drawings were 
rare. Vesalius published his new anatomy, called On the Fabric of the 
Human Body in 1543, the same year as Copernicus' On the revolution 
of the celestial spheres, the revolutionary book about the solar system. 
Both works became famous and hotly debated. They are milestones 
in the history of science. Source : Masquelet, 1986. Bottom : These 
remarkable ink drawings of the exposed brain are attributed to the 
great painter Titian, who illustrated Vesalius' classic anatomy, which 
was a true work of art as well as science. Notice that the two cortical 
hemispheres on the right have been separated to show the corpus cal¬ 
losum, the great fiber bridge running between the two halves, with 
some 100 million neuronal axons. To the early anatomists it appeared 
to be a tough or calloused tissue, and was therefore called the 'cal¬ 
loused body', corpus callosum in Latin. Source : Squire et al., 2003. 

understand a knotty three-dimensional object from 
all perspectives, it helps to hold your own hands 
in front of you to represent the two hemispheres to 
remind yourself. It is essential to know the major 
lobes and other major brain divisions, just as it is to 
know the names of the continents in earth geography. 
Throughout this text, we will be presenting brain stud¬ 
ies and relating their findings to our understanding of 
human cognition. These basic brain figures will serve 
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as a guide for interpreting the data of neuroimaging 
studies that are provided throughout this text. 

4*0 SOME HISTORY, AND ONGOING 
DEBATES 

The idea of the brain as the source of our experi¬ 
ences goes back many centuries, as shown by the 
quotation at the beginning of this chapter (attrib¬ 
uted to Hippocrates some 2500 years ago). But care¬ 
ful, cumulative study of the brain really began with 
the Renaissance. The Antwerp anatomist, Andreas 
Vesalius, was the first to publish a detailed atlas of the 
human body, including the brain, in 1543. Before that 
time, religious and legal prohibitions made it a crime 
to dissect human cadavers. A century later, a famous 
Rembrandt painting called The Anatomy Lesson of 
Dr. Tulp' shows the sense of wonder felt by physicians 
to be able to actually see the human body in detail 
(see Figure 1.1). Leonardo da Vinci made sketches 
of the human skull and body at about the same time 
(1490-1510). The Renaissance was interested in every¬ 
thing human, and the effort to understand the brain 
grew as part of that broad sense of curiosity. 

Today's research is deeply rooted in the history of 
science. The behavioral sciences date their beginnings 
to the 19th century. But careful brain studies go to the 
very beginnings of modern science in the European 
Renaissance, the time of Galileo, Copernicus, Newton, 
and Descartes. Color perception really began to be 
studied with Newton's prism experiments in 1665. The 
invention of the light microscope by Leeuwenhoek 
and others in the 1600s leads straight to discover¬ 
ies about nerve cells and their properties by Santiago 
Ramon y Cajal in the 19th and 20th centuries. 

One pattern in the history of science is that the 
more we learn, the more we can see simple and gen¬ 
eral patterns in the evidence. We will bring out these 
simplifying principles throughout the book. 

The Renaissance origins of brain science are clearly 
apparent even today. Brain terminology is based on 
Latin, the international language of science for many 
centuries. We still talk about the occipital, temporal, 
parietal, and frontal lobes of the cortex, all Latin words. 
Because early studies were done with the naked eye 
or with simple microscopes, most were named for the 
way they looked visually. Thus, the thalamus means 


the 'bridal chamber', the amygdala is the 'almond', cor¬ 
tex means 'outer bark', and so on. Practically all brain 
terms use everyday Latin words. That fact will simplify 
your understanding of anatomical terms, and we will 
mention the origins of each term when it is introduced. 

The human brain evolved over some 200 million 
years from early mammalian origins. It is very com¬ 
plex, even the parts that can be seen with the naked 
eye. Generations of scholars have contributed to its 
study. For example, Rene Descartes (Figure 1.10) is 
known today mostly as a mathematician and philos¬ 
opher. But he was also a careful student of the brain. 
In one famous observation he obtained the eye of an 
ox, scraped off the tissue from the back of the eyeball 
to expose its tough white outer shell, the sclera, and 
showed that light images coming through the lens 
of the eye were projected onto the white sclera like a 
screen, so that one could see projected images by point¬ 
ing the eyeball at a well-lit object. It astonished many 
people that the visual stimulus was projected upside- 
down on the back of the eye. Descartes was able to show 
that this is a direct result of the optics of the lens 1 . 

4*1 The mind and the brain 

Descartes is often considered to be the originator of 
modern mind/body philosophy. The basic question 
seems simple: is the world basically mental or physical? 
Or, in today's language, can your conscious experience 
be explained by neurons? Perhaps nerve cells them¬ 
selves are just ideas in the minds of scientists. The brain 
basis of consciousness has now become mainstream in 
cognitive neuroscience (Edelman, 1989; Palmer, 1999; 
Koch, 1996; Tulving, 2002; Baars et al, 2003a). Numerous 
articles have appeared in the last fifteen years. In July of 
2005, Science magazine listed 'the biological basis of con¬ 
sciousness' as one of the top questions in science today. 
Nobel-winning scientists like Francis Crick, Gerald 
Edelman and Herbert Simon have devoted years of 
effort to the question. 

In everyday language we constantly switch back 
and forth between the language of mind and brain. We 
take a physical aspirin for a mental headache. We walk 
to the physical refrigerator because we experience a 
mental craving for ice cream. Do conscious experiences 
'cause' physical actions, or vice versa? Common sense 
doesn't care. It just jumps back and forth between 
the discourse of mind and body. But things get 


1 It still baffles many people that we do not experience the visual world upside-down, since that is how the image is projected on the retina. 
What do you think is the answer? 
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FIGURE 1.10 Descartes: philosopher, mathematician, brain explorer. Rene Descartes (left) and his figure showing the optics of the eyes 
(right). Because Descartes was convinced that the soul or psyche was a unified whole, he rejected the idea that paired structures of the brain 
could support the soul. But almost all of the brain looks doubled to the naked eye: two hemispheres, two eyes and ears, two subcortical halves, 
and two sides of the cerebellum. Descartes therefore decided that the tiny pineal gland, which looks like a tiny dot to the naked eye, must be 
the point of connection between the divine soul and the earthly body. Unfortunately for Descartes, microscopic studies after the 17th century 
showed that the pineal gland also has bilateral symmetry, just like the rest of the brain. Source: Bennett, 1999. 


complicated when we try to think more carefully. In 
the physical realm of aspirins and refrigerators, ordi¬ 
nary causality explains how things happen. Ice cream 
melts in the sun and aspirins dissolve in water. They 
follow the laws of physics and chemistry. But mental 
events are affected by goals , emotions , and thoughts , 
which seem to follow different laws. Ice cream does 
not melt because it wants to - but humans eat ice cream 
because they want to. Human language has thousands 
of words to describe desires and experiences, but those 
words do not apply to physical objects. 

As we will see, for the first time, we have a large 
body of empirical evidence that has a direct bearing on 
the question of conscious cognition. Some brain regions, 
like the Ventral visual stream', are widely believed to 
support conscious contents, the visual events that peo¬ 
ple can easily report. There is good evidence, however, 
that not all brain regions support conscious, reportable 
events. For example, the dorsal visual stream enables 


hand-eye coordination in reaching for an object, but 
there is good evidence that by themselves, these brain 
areas do not support conscious contents (Goodale 
et al. r 1991). Similarly, it is believed that brain regions 
like the cerebellum do not support conscious experi¬ 
ences. There is growing agreement that the relationship 
of mind to brain is an empirical, testable question. 

4*2 Biology shapes cognition and emotion 

Human emotions turn out to have deep biological roots 
in the mammalian brain, regulated by more recent 
layers of the neocortex. As a result, the evolution¬ 
ary history of the human brain, going back some two 
hundred million years to early mammals, is relevant 
to a host of important questions. Maternal love and 
infant attachment now seem to be rooted in an ancient 
mammalian structure called the peri-aqueductal 
gray matter or PAG. Vocalizations of distress and 
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FIGURE 1.11 Charles Darwin and the biology of mind. Many 
people in the centuries after Rene Descartes were fascinated by 
the conscious mind and its relation to the brain. Charles Darwin, 
for example, wrote a book called The Expression of Emotions in Man 
and Animals. Darwin thought about human emotions as biologi¬ 
cally based - which was not meant to minimize our vast cultural 
and individual influences, of course. The picture shows Darwin as 
a young man, around the time of his historic voyage to the Pacific 
Ocean on The Beagle. Source: Finkelstein, 2000. 

attachment seem to have an evolutionary connection 
to the emergence of speech prosody - the sing-song of 
language - and even to music. Thus, biology turns out 
to have relevance in many different ways. 

More than anyone else, Charles Darwin helped to 
establish the biological context of the human species. 
Darwin made numerous observations about emotional 
expressions in animals and humans (Figure 1.11). For 
hundreds of years people must have noticed the simi¬ 
larities, but in Europe and other regions it was essen¬ 
tially taboo to point out those similarities. There is 
now good evidence about the emotional brain regions 
that humans share with other mammals (Panksepp, 
2005). In Darwin's time that idea was very controver¬ 
sial. Today, Darwin's book The Expression of Emotions in 
Animals and Man is considered a classic (Ekman, 2003). 

Darwin would never deny the importance of cul¬ 
ture and environment. Humans are the most adaptable 
species we know, the one most capable of learning new 
things by way of cultural transmission. The 'nature- 
nurture' controversy is not really a divisive debate any 



FIGURE 1.12 Hermann von Helmholtz was an amateur who 
ended up making historic contributions to science. Helmholtz was 
one of the first to propose that the visual system makes "unconscious 
inferences' that go far beyond the raw input from the eyes. That idea 
was so controversial that he was forced to withdraw it, but it is a 
standard notion in sensory science today. Source: Bennett, 1999. 

more. Most scientists see human behavior as combin¬ 
ing biology, culture, and environment (see Chapter 15). 

Helmholtz was a great sensory physiologist and 
psychologist in the 19th century, whose works on 
vision and audition are still read (Figure 1.12). But his 
biggest claim to fame is the physical law of conserva¬ 
tion of energy, which he demonstrated using electri¬ 
cal stimulation of dissected frog's legs. The insight that 
electricity was part of nervous activity led to the dis¬ 
covery of nerve potentials and the electrical activity 
of the brain in the early 20th century. Today, the elec¬ 
tromagnetic activity of the brain gives us a whole set 
of imaging methods: EEG, evoked potentials, single¬ 
neuron recording, MEG, and even TMS, magnetic stimu¬ 
lation of the brain (Chapter 4). Indeed, all brain-imaging 
tools make use of fundamental physics and chemistry. 

In Helmholtz' time it was well known that an elec¬ 
trical pulse would cause a pair of frog's legs to contract. 
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FIGURE 1.13 A schematic of Helmholtz's apparatus for meas¬ 
uring the time course of muscle contraction and the propagation 
velocity of the nerve impulse. Source: Bennett, 1999. 


as if the frog were jumping. The idea that the brain and 
body are pervaded with electricity caught the popular 
fancy, leading to cult ideas like 'animal magnetism'. 
Helmholtz' conservation of energy experiment in frogs' 
legs was a major step towards a naturalistic conception 
of the brain and body (Figure 1.13). 


43 CajaPs neuron doctrine: the working 
assumption of brain science 

Santiago Ramon y Cajal (Figure 1.14) is credited with 
the neuron doctrine, one of the founding assumptions of 
brain science, stating that 'the nervous system consists 
of numerous nerve units (neurons), anatomically and 
genetically independent'. It is important to realize that 
the idea that the body is composed of cells dates back 
only to 1839, with the use of light microscopes and 
tissue-staining techniques that showed the details of 
living cells for the first time in human history. 

It became thereby possible to prove that all body 
tissues consisted of cells, except for nervous tissue. 
The reason was that neurons are immensely branched, 
and their axons and dendrites spread so thickly, and 
make such close contact with other nerve cells that it 
was impossible to tell whether they actually touched 
or not. Thus a controversy arose over the nature of the 




FIGURE 1.14 Left: Santiago Ramon 
y Cajal, founder of brain science. Golgi 
color stains were used by Santiago 
Ramon y Cajal, perhaps the most 
important early pioneer in neuro¬ 
science, to bring out basic facts about 
nerve cells under the light microscope. 
Cajal showed the microanatomy of neu¬ 
rons. Right: His beautiful illustrations of 
his microscopic observations. Today's 
methods for studying neuronal micro¬ 
structure are advanced versions of the 
Golgi-Cajal approach (see Figure 1.16). 
Source: DeFelipe, 2002. 
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FIGURE 1.15 Cajal 's first draw¬ 
ing of a microscopic slice of nerve tis¬ 
sue, from the cerebellum of a hen. Cajal 
made use of the Golgi staining method, 
which caused small numbers of nerve 
cells to stand out from the surrounding 
tissue. Source : DeFelipe, 2002. 


nervous system - whether it consisted of billions of 
cells, or whether it was essentially one great continu¬ 
ous network. 

In Madrid, Santiago Ramon y Cajal used the Golgi 
stain, which brought out a small sample of neurons 
in a slice of tissue in such a way that the shape of the 
cells could be seen (Figure 1.15). Cajal was also able 
to prove that axons end in free terminals, and must 
therefore be separate cells. 

The nerve impulse - the electrochemical 'spike' 
that travels down the axon to its terminals - was first 
demonstrated in the giant axon of the squid. Because 
nerve cells are found in all animals, and their basic 
properties appear to be conserved among many spe¬ 
cies, the squid axon is an important test-bed for other 
classes of neurons. That discovery was made in 1939, 
a century after the cell doctrine was proposed. Ten 
years later it was found that sodium and potassium 
ions (Na+ and K+) rapidly move in and out of nerve 
axons to propagate the axonal spike and, in 1952, 
Hodgkin and Huxley (1952) constructed the model of 
the action potential that we use today (Figure 1.16). 
About the same time, it became possible to observe 
the even tinier synapse through the use of electron 
microscopes, down to several hundred micrometers. 

There is an ironic ending to the Cajal story, however. 
The great controversy about whether neurons are sep¬ 
arate continues today. Cajal seemed to have settled the 
question forever when he showed that axons had free 
nerve endings. However, recent evidence shows that 
many points of transmission between nerve cells are 
not chemical synapses but electrically continuous tis¬ 
sue called gap junctions. Some neurons are nestled 
against each other, without the classical synaptic gaps. 



FIGURE 1.16 A modern version of Cajal's figure. Neurons 
in a modern X-ray micrograph using a chemical stain. Source : 
Standring, 2005. 

It is, therefore, possible that some neurons do form a 
continuous structure, though there is no question that 
chemical synapses exist in the trillions. Like so many 
other scientific controversies, the debate goes on at 
ever more refined levels. 
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FIGURE 1.17 Pierre-Paul Broca, who defined expressive 
aphasia. He was the first to make a convincing case for a single, 
highly specialized function in a specific location in cortex. Source : 
Aminoff and Daroff, 2003. 


4.4 Pierre-Paul Broca and the localization of 
speech production 

Medical observations are also relevant to the mind 
and brain sciences. It was the French physician, Pierre- 
Paul Broca (Figure 1.17), who first discovered a region 
of the left hemisphere that was unquestionably tied 
to language production, a higher mental function. The 
'speaking' region of the left hemisphere is therefore 
called Broca's area. 

Controversy has raged over the question of brain 
localization - whether specific regions of the brain 
serve very specific functions. Broca was the first to 
establish firmly a specialized region of the higher brain 
in 1861, based on a patient who suffered from epilepsy. 
Broca's patient had lost all ability to speak except the 
single word 'tan'. After the patient died, Broca was able 
to perform an autopsy, finding 'damage to the pos¬ 
terior part of the third frontal convolution in the left 
hemisphere'. The brain of Broca's patient has been pre¬ 
served and you can see the location of the damage in 
the frontal region of the left hemisphere. (Figure 1.18). 
He concluded that this part of the frontal lobe was 
indispensable for speech production. Six months later, 
Broca presented a similar case, again with damage to 
this part of the left frontal lobe. Despite criticisms that 



FIGURE 1.18 The brain of Broca's early patient has been preserved. If you look at the frontal region of 
the left hemisphere you can see a large hole in 'Broca's area'. Source: Ramachandran, 2002. 
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other areas of the brain were involved in his cases and 
that some patients with similar disorders did not have 
frontal lobe lesions, Broca's observations came to be 
accepted because of the weight of subsequent evidence. 
Today, Broca's area in the left frontal lobe is widely rec¬ 
ognized as a critical component of the language region 
(Aminoff and Daroff, 2003). 

Much of what we know about the human brain 
was first discovered from very careful study of spe¬ 
cific deficits. That effort continues today. Another 
important finding for language was made by Carl 
Wernicke (Figure 1.19), who published a monograph 
in 1874 describing a model for language based on his 
work in human pathology that remains a key aspect of 
cognitive neuroscience today. Today, Wernicke's area 
in the left upper part of the temporal lobe is widely 
recognized as an important brain area for recep¬ 
tive language. Patients with brain damage in this 
region and deficits in speech comprehension are still 
called 'Wernicke's aphasics'. However, with better 
techniques of brain imaging, these areas are turning 
out to be more complex and flexible than they were 
thought to be. 



Here is a speech sample from a modern patient 
(FL) with receptive aphasia, due to damage in or near 
Wernicke's area (Dronkers and Ogar, 2003): 

Examiner: Tell me what you think your problem is. 
FL: Ah, where do I start the tesseinemen from? They 
tell me that my brain, physically, my brain is perfect, 
the attitudes and everything is fine, but the silence 
now, that I have to reeh-learn through edgit again, 
physically nothing wrong with it. It's perfect the doc¬ 
tors tell me. They have attitude. Physically I have loo- 
zing absolute nothing on my head, but now I have to 
go through these new attitudes to looalize how, some, 
how can I say to? Some what that I can reeh-learn 
again so I can estep my knowledges, so th'you kyou 
again, what how can I say that y . . . 

Notice that this patient is much more fluent than Broca's 
classic patient of 1861 who could only pronounce one 
syllable. However, this patient's ability both to produce 
and comprehend meaningful speech is impaired. Using 
brain imaging, the location of focal damage is routinely 
determined today so that diagnosis is no longer depend¬ 
ent only on neurological inferences and autopsies. 


C 



FIGURE 1.19 Wernicke and the comprehension of speech. Some years after Broca's work 
on speech output , the German physician Carl Wernicke (left panel) discovered a part of the brain 
involved in speech input - perception and comprehension. Wernicke studied a variety of aphasic 
patients (a-phasia, meaning 'not speech'), and concluded that damage in different cortical locations 
led to different disorders. He distinguished between semantic aphasias and receptive and productive 
types. The right panel shows the 'house' model (because it looks like a simple drawing of a house) 
for language areas from the 19th century. The label A refers to the auditory language area for com¬ 
prehending speech, B refers to motor language area for producing speech, and C refers to the dis¬ 
tributed representations of concepts throughout the brain. Source: Wernicke and Koehler, 2003. 
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FIGURE 1.20 Conduction aphasia. Careful anatomical dis¬ 
sections showed a fiber bundle connecting Broca's and Wernicke's 
areas of the left hemisphere. Based on this evidence, Wernicke was 
able to predict a new language deficit in 1874 called disconnection 
aphasia. If the fiber bundle between Broca's and Wernicke's areas 
were damaged, he thought, patients should have difficulty transfer¬ 
ring information from the receptive area to the production region. 
That prediction was borne out and this type of deficit is called con¬ 
duction aphasia. Source : Dronkers and Ogar, 2003. 

It was well known in the 19th century that most 
of the mass of the cortex is white matter, consisting 
of billions of axons emerging from the gray cell bod¬ 
ies in the surface layers of the cortex. Those axons 
are wrapped in the white protective and supportive 
myelin cells, filled with fat-like lipid molecules. They 
therefore look white to the naked eye. Most of the 
white matter therefore consists of great fiber bundles, 
connecting every region of cortex to every other, like 
some great highway system. Careful anatomical dis¬ 
sections showed a fiber bundle connecting Broca's 
and Wernicke's areas in the left hemisphere. These 
fiber bundles are known as arcuate fasciculi, Latin for 
'arched little bundles'. (Again, an everyday Latin 
term has become a long and complicated-sounding 
word to our ears.) 

Based on this evidence, Wernicke was able to pre¬ 
dict a new language deficit in 1874 called disconnection 
aphasia. If the fibers between Broca's and Wernicke's 
areas were damaged, he thought, patients should have 
difficulty repeating speech sounds - transferring infor¬ 
mation from the receptive area (Wernicke's) to the pro¬ 
duction region (Broca's). That prediction was borne 
out (see Figure 1.20). It is believed that there are a 
number of such disconnection syndromes, with some¬ 
what different symptoms from lesions that involve 
damage to the outer cell bodies of cortex. A number of 
other types of aphasia are thought to exist. 


Notice that these language areas usually exist in the 
left hemisphere (Figure 1.21). Modern evidence shows 
that the right side of the brain does comprehend 
language, but it does not control vocal output. The 
right side is believed to be sensitive to the emotional 
content of language, such as humor and irony. Left¬ 
side dominance seems to apply to about 90 per cent 
of the population. About 10 per cent of normal people 
have right hemisphere dominance for language. 

William James (Figure 1.22) is best known today 
as a psychologist and philosopher, however, he also 
trained to be a physician and painter. James was first 
hired at Harvard University to teach brain anatomy, 
and his 1890 description of the brain is in good agree¬ 
ment with our understanding today. Note his draw¬ 
ing showed Broca's and Wernicke's areas based on the 
medical evidence from aphasias (Figure 1.23). 

James' Principles of Psychology is often consid¬ 
ered the best summary of 19th century psychology 
and brain science in the English language. Among 
many other points, James discussed the brain studies 
of a Dr. Mosso, who observed that the blood supply 
to active brain regions increased (Figure 1.24). That 
finding is the basis of modern brain imaging methods 
based on hemody-namics (the flow of blood), including 
PET scans and MRI (Figure 1.25). 

[In Mosso's experiments] the subject to be observed 
lay on a delicately balanced table which could tip 
downward either at the head or at the foot if the 
weight of either end were increased. The moment 
emotional or intellectual activity began in the sub¬ 
ject, down went the balance at the head-end, in con¬ 
sequence of the redistribution of blood in his system. 
(William James, Principles of Psychology (1890)) 

Thus, 19th century scientists discovered localized 
brain functions for language. They were not only 
concerned with the classic psychological aspects of the 
mind, but also with their relation to the brain. 

4.5 The conscious and unconscious mind 

Nineteenth century physicians believed that human 
conscious experience depended on the cortex, the 
outer layer of the brain. James wrote: 

But is the consciousness which accompanies the activ¬ 
ity of the cortex the only consciousness that man has? 
or are his lower centres conscious as well?. . . the cor¬ 
tex is the sole organ of consciousness in man. 

Contemporary neuroimaging evidence tends to 
support James' claim, although some scientists believe 
that subcortical regions are also involved (see Chapters 
2 and 8). The question of conscious perception 
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FIGURE 1.21 Language regions of the left hemisphere, studied by fMRI (functional magnetic 
resonance imaging). Red areas are involved in speech production , while yellow ones show up in speech 
perception and comprehension. The classic Broca's and Wernicke's areas are marked by circles. Notice the 
close similarity between the modern understanding of language regions and the 19th century account. 
(PAC = Primary Auditory Cortex, the initial region in the cortex for second processing; ant. STS = anterior 
superior temporal sulcus, the front part of the upper fold of the temporal lobe.) Source : Frackowiak, 2004. 


and cognition has again become a major focus of 
research, as we will see. 

Nineteenth century scientists were profoundly inter¬ 
ested in the question of consciousness. Ramon y Cajal 
proposed the idea of a 'psychic neuron' in the cortex, 
a type of neuron that would support conscious expe¬ 
riences. The beginning of psychophysics in the early 
19th century was very much inspired by an effort 


to solve the mind-body puzzle, and the pioneering 
psychophysicist, Gustav Fechner, claimed that he had 
found the answer by showing a mathematically exact 
relationship between subjective sensory intensity and 
physical stimulus intensity. At the end of the 19th cen¬ 
tury, William James proclaimed that 'Psychology is the 
science of mental life', by which he meant conscious men¬ 
tal life (James, 1890/1983, p. 15). James was not alone. 
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FIGURE 1.22 William James taught brain anatomy. While 
William James is best known today as a psychologist and philoso¬ 
pher, he also trained to be a physician and artist. James was first 
hired at Harvard University to teach brain anatomy and his 1890 
description of the brain is in good agreement with our understand¬ 
ing today. Source : Courtesy of the National Library of Medicine. 



FIGURE 1.23 James showed Broca's and Wernicke's areas based 
on the medical evidence from brain damage. Source: James, 1890. 

About that time, some began to disagree. Scientists 
like Helmholtz, Loeb, and Pavlov advocated a more 
physicalistic view of mental life. After 1900, Pavlov 
became famous for his experiments on classical condi¬ 
tioning in dogs. This helped to convince many psychol¬ 
ogists that ultimately all behavior could be explained 
in terms of simple behavioral units, based on reflexes 
(Figure 1.26). In the USA, John B. Watson was the first 
person to make radical behaviorism famous, arguing 
that any reference to consciousness was improper, 
since we can only publicly observe physical behavior 
and the physical brain. Watson's famous slogan was 





FIGURE 1.24 William James (1890) described the research 
of Dr. Mosso, who found a way to measure blood pressure during 
demanding mental tasks. Mosso's work anticipated current meas¬ 
ures of brain blood flow like fMRI. Source: James, 1890. 



FIGURE 1.25 Functional MRI measures local blood flow 
changes in the brain. Contemporary fMRI experiments are based 
on blood flow changes in the brain whenever some brain regions 
require more oxygen and glucose. It is a modern version of 
Dr. Mosso's 19th century discovery. Source: Thomas Ramsoy. 
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FIGURE 1.26 Pavlov operating with three colleagues on 
an anaesthetised dog in the department of Physiology, Imperial 
Institute for Experimental Medicine, St. Petersburg, c. 1902. 
Source : Tansey, 2006. 


that 'consciousness is nothing but the soul of theol¬ 
ogy', and therefore unscientific. For much of the 20th 
century human consciousness was avoided by scien¬ 
tists. Some saw it as too burdened with philosophical 
questions, too difficult to test experimentally or too 
subjective to be studied scientifically. 


There were some practical reasons for this. It is dif¬ 
ficult in many cases to be sure about someone else's 
experience, so that it can be hard to repeat experi¬ 
ments in the way reliable science requires. 


FRONTIERS OF COGNITIVE NEUROSCIENCE 


A biological theory of consciousness 



FIGURE 1.27 Gerald M. Edelman, MD, PhD, The 
Neurosciences Institute, San Diego, CA, USA. 

We all experience a stream of conscious experiences — 
our inner speech, thoughts and feelings, our sense of who 


we are. Conscious knowledge of ourselves and the world 
emerges in the first few years of life. It is something we 
share with millions of others, yet it is very private: no one 
else sees, feels, or understands our personal conscious 
viewpoint exactly as we do. Awareness is omnipresent in 
humans and yet intimate and personal. At the same time 
we know there are regions of the brain, like the cerebel¬ 
lum, that do not give rise to conscious experiences. So 
what is the difference between brain regions that result 
in conscious reports and those that do not? (Chapter 8) 

The brain bases of consciousness is a core topic 
of the mind-brain sciences. A leader in this quest is 
Dr. Gerald Edelman, Nobel Laureate and Director of the 
Neurosciences Institute, and Chair of the Department 
of Neurobiology at the Scripps Research Institute in La 
Jolla, California. Edelman also has a lifelong professional 
interest in classical music and performance. Gerald 
Edelman has developed a biological theory of brain func¬ 
tion called Neural Darwinism, with a central role for 
conscious contents, which continues to be developed (see 
Chapter 8). 

Neural Darwinism is a large-scale theory of brain 
development and function. It suggests that brain devel¬ 
opment and brain dynamics are selectionist in nature, not 
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FIGURE 1.28 The three main tenets of Neural Darwinism. Developmental 
selection yields a diverse set of neural circuits by cell division, cell death, and the 
extension of neuronal branches, the dendrites and axons, to reach out to others and 
build synaptic connections. In the next stage, experiential selection leads to changes 
in synaptic strengths favoring some pathways over others. Reentrant connections 
enable coordination of neural activity between different regions of the brain. (Black 
dots indicate strengthened synapses.) Source : Squire, 2004. 


instructionist, as in the case for digital computers, which 

carry out explicit symbolic instructions (see Figure 1.28). 
Selectionist systems have four features: 

1. A set of elements that are characterized by diver¬ 
sity, like the members of a species, antibodies in the 
immune system, or neurons in the brain. 

2. These elements can reproduce or amplify. 

3. A process of selection operates on the products of 
diversity. In evolution, selection favors some offspring 
in the species and not others. In the immune system, 
selection operates on successful antigen-antibody 
matches. The brain has two selectional repertoires, one 
developmental, with the neurogenesis of a vast body of 
neurons, and a connectional repertoire as neurons make 
synaptic connections and prune all but the successful 
ones to produce the mature brain. The repertoire of 
connections continues to adapt and grow throughout 
our lives. 

4. Finally, selectionist systems show degeneracy, the 
ability to use different combinations of elements to 


perform the same function (Edelman & Gaily, 2001). 
Degeneracy applies to all levels of life. We have two 
lungs, two halves of the brain, and a great many 
redundant pathways at all levels of biological organi¬ 
zation. Degeneracy makes it possible to overcome 
damage and disease by using alternative pathways 
and organs. It can enable living things to survive 
setbacks. 

Conscious experiences 

According to Edelman, conscious experiences result 
from reentrant interactions among neurons in the 
thalamocortical system, the core of the mammalian 
brain. Reentry is the constant resonant activity between 
two ensembles of neurons, which serves to modify the 
connections between them, and also to select the mem¬ 
bers of the ensembles themselves. It is a highly adaptive 
process, comparable to social networking, where the 
networks continue to expand and develop depending 
on person-to-person interactions. In this view, a subset 
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of thalamocortical neurons work together in a large 
"dynamic core" of interacting neurons. The dynamic 
core as a whole can change from moment to moment, to 
adapt to new conditions, retrieved memories, and goals. 
Primary consciousness concerns the perceptual world 
as well as memory; higher-order consciousness extends 
beyond primary consciousness to language, abstraction, 
and thought. 

One property of consciousness is its extraordinary 
range of contents — sensory perception, visual imagery, 
emotional feelings, inner speech, abstract concepts, and 
action-related ideas, to name only some. The broad range 
of contents reflects the fact that consciousness involves 
multiple brain regions. Sensory cortex clearly contributes 
to conscious perceptual events, but there is evidence that 
nonsensory regions like the frontal and parietal cortex 
are involved even in sensory experiences, like the sight of 
a coffee cup. An integrative view of consciousness must 
also include interactions among brain regions involved 
in conscious recall, conscious control of motor skills, and 
the like. Edelman and colleagues have suggested that the 
workings of the dynamic core could handle such a wide 
range of brain activities. 

This broad outline has been filled in by biologically 
anchored modeling of the thalamocortical system, the 
hippocampus, and other regions. Some of these models 
run robots in realistic environments, to see if they can 
learn in ways that resemble mammalian learning. 

Edelman and Tononi (2000; Tononi & Edelman, 1998) 
suggest that the content of any conscious event is not 
simply a consequence of activity in the relevant cortical 
area. The sensation of redness does not come from activ¬ 
ity in the visual cortex alone, although such activity may 
be required. Rather, a sensation of redness is determined 
by the state of the entire dynamic core. More formally, 
any conscious event can be represented by a single point 
in an N-dimensional space, where N represents the 
number of neuronal groups that are part of the dynamic 


core at any time. A sensation of redness is therefore 
entailed by a very high dimensional discrimination — 
not just a discrimination among possible colors, but 
rather occurs among a vastly larger number of possible 
dynamic core states. 

According to the extended theory of Neural Dar¬ 
winism, the diversity of conscious contents arises from 
the ways in which different neuronal groups influence 
the N-dimensional space. Although a large proportion 
of the mammalian cortex is surprisingly uniform in its 
histology, input to different cortical areas varies greatly. 
For example, visual input is very different in its statis¬ 
tical structure from proprioceptive input. Neuronal 
groups sensitive to color may therefore organize the 
space in a way different from those sensitive to proprio¬ 
ception. Each sensory modality will therefore influence 
the state of a dynamic core in a unique manner, and this 
may explain why the content of a conscious event domi¬ 
nated by vision is reliably different from one dominated 
by a sound, a touch, or a taste. 
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5*0 THE RETURN OF CONSCIOUSNESS 
IN THE SCIENCES 

In the 1970s, many psychologists became dissatisfied 
with behaviorism and began to pursue a different 
path. While using behavioral measures in the labora¬ 
tory, cognitive psychologists were interested in mak¬ 
ing inferences from those observations. We will see 
one prominent example in this chapter, the idea of 


working memory. There are many others. Cognitive 
psychologists have brought back visual imagery, vari¬ 
ous types of memory, unconscious (implicit) cogni¬ 
tion, and many other terms. However, these concepts 
are always studied behaviorally in the laboratory 
under very closely controlled conditions. The concepts 
of working memory, imagery, and so on, are inferred 
from behavioral observations. They are theoretical 
explanations, much as electrons are in physics. No 
one has ever seen an electron, but the concept of an 
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BOX 1.2 The mind-brain debate today 


Philosophers continue to debate whether conscious expe¬ 
riences can be understood in terms of the brain. However, 
many scientists today believe that it has become a pro¬ 
ductive scientific question again. Patricia Churchland has 
been among the leading philosophers coming back to a 
naturalistic approach to the mind and brain. She recently 
said that 'neuroscientific data are relevant to longstand¬ 
ing, traditional problems in philosophy: the nature of 
consciousness, the self, free will, knowledge, ethics and 
learning'. 

Psychologists have studied sensory processes since 
the early 1800s. We experience our world through our 
senses: we have the feeling that our mind-brain is intri¬ 
cately involved with the interplay between our physical 
world and our brain. We readily get help for our percep¬ 
tual abilities by obtaining eyeglasses from optometrists, 
hearing aids from audiologists. And we spend time tun¬ 
ing the objects that provide us with some of our sensory 
inputs, such as car speakers and audio equipment. Yet 
the correspondence between our visual and auditory per¬ 
ception of the world and the physical properties of that 
world is still not entirely known. Clearly, the mind and 
brain have a sense of sights and sounds within our envi¬ 
ronment, yet mapping the path between the physical and 
the perceived remains a mystery. 

It seems that words like 'mind' and 'brain' represent 
different sources of information. If we look out at the 
world from within our brains (as we are always doing), 
things seem different than if we look at our brains from 
the outside, in a brain scanner, for example. 

When a computer shows us how much memory it has, 
no one thinks that it has just discovered a new realm of 


existence. When people tell us about their sensory experi¬ 
ences, they have not done so either. But they have told 
us something valuable. Philosophical journals are still 
filled with mind/body debates, but scientists must define 
questions in empirical terms. Today, we are increas¬ 
ingly confident about the evidence from both mental 
and brain perspectives, and we are beginning to see how 
the two points of view converge when we study them 
carefully. 



FIGURE 1.29 Patricia Churchland has been among the 
leading philosophers coming back to a naturalistic approach 
to the mind and brain. Source : Patricia Churchland, UCSD. 


electron explains many different kinds of observable 
phenomena (Baars, 1986). 

Perhaps the greatest change over the last twenty years 
within cognitive psychology and cognitive science 
more generally, has been the acceptance of conscious¬ 
ness as a legitimate and tractable scientific problem. 
During much of the twentieth century the field was 
enmeshed in the philosophical tangles of the body- 
mind problem, and stifled by the empirical limitations 
of introspectionism. More recently, it has gradually 
become clear that neither of these represent the sort 
of fundamental obstacle that was originally feared. 
Furthermore, the need to account for phenomena such 
as blindsight and implicit memory, in which percep¬ 
tion and recall were clearly proceeding in ways that 
were at variance with the conscious experience of 
the perceiver or the rememberer, argued strongly for 
the need to bring back the study of conscious aware¬ 
ness into the empirical psychological fold. (Baddeley, 
personal communication) 


In recent years, the reluctance to study consciousness 
has begun to fade. Many cognitive psychologists study 
both explicit (conscious) and implicit (unconscious) 
processes. As we will see, there is now a large body of 
evidence that our perception of the world around us 
is partly unconscious, although the result of the per¬ 
ceptual process is conscious. Many aspects of memory 
are unconscious, while episodic memory involves the 
record of conscious events in the past. With the advent 
of functional imaging technology, it has become pos¬ 
sible to make careful comparisons between brain 
events involving conscious versus unconscious cog¬ 
nition. Currently, about 5000 articles per year refer 
to consciousness and its many synonyms. The syno¬ 
nyms include 'awareness', 'explicit cognition', 'epi¬ 
sodic recall', and 'focal attention'. Those terms are 
defined experimentally by measures of 'accurate 
report'. We simply ask people if they perceived or 
recalled an event, and then try to check the accuracy 
of their reports. In that sense, the different synonyms 





5.0 THE RETURN OF CONSCIOUSNESS IN THE SCIENCES 


27 


TABLE 1.3 Commonly studied conscious and unconscious brain events 


Conscious Unconscious 


1 Explicit cognition 

2 Immediate memory 

3 Novel informative and significant events 

4 Attended information 

5 Focal contents 

6 Declarative memory (facts etc.) 

7 Supraliminal stimulation 

8 Effortful tasks 

9 Remembering (recall) 

10 Available memories 

11 Strategic control 

12 Grammatical strings 

13 Intact reticular formation and bilateral 
intralaminar thalamic nuclei 

14 Rehearsed items in working memory 

15 Wakefulness and dreams (cortical arousal) 

16 Explicit inferences 

17 Episodic memory (autobiographical) 

18 Automatic memory 

19 Intentionallearning 

20 Normal vision 


Implicit cognition 
Longer term memory 

Routine, predictable and non-significant 
events 

Unattended information 
Fringe contents (e.g. familiarity) 
Procedural memory (skills etc.) 
Subliminal stimulation 
Spontaneous/automatic tasks 
Knowing (recognition) 

Unavailable memories 
Automatic control 
Implicit underlying grammars 
Lessened reticular formation or bilateral 
intralaminar nuclei 
Unrehearsed items 

Deep sleep, coma, sedation (cortical slow 
waves) 

Automatic inferences 
Semantic memory (conceptual 
knowledge) 

Noetic memory 
Incidental learning 
Blindsight (cortical blindness) 


for conscious events are assessed in the same way, 
and therefore seem to have a basic similarity. We will 
call them 'conscious', but also use the technical terms 
(see Baars et al., 2003). 

In the 19th century, figures like Sigmund Freud and 
William James were deeply interested in understanding 
the relationship between mind and brain. Freud began 
his medical career in neurology, and even developed an 
early neural network model. Early in his career he dis¬ 
covered a new chemical stain - gold chloride - which 
allowed certain neurons to stand out clearly under the 
microscope. The first such stain was discovered by 
Camillo Golgi in 1873, and revolutionized the ability 
to observe nerve cells under the light microscope (see 
http:/ / nobelprize.org/medicine/articles/golgi/). 

5*1 How conscious and unconscious brain 
events are studied today 

Psychologists and neuroscientists have devised a very 
wide range of methods for studying conscious and 


unconscious processes in the brain. Table 1.3 shows 
a subset of all the methods currently available. They 
all have pros and cons, of course. Some have impli¬ 
cations for episodic memory (memory for conscious 
events), while others show that visually guided reach¬ 
ing for objects is largely unconscious. There are many 
unresolved questions, whether 'conscious' cognition is 
the best terminology, whether selective attention must 
precede conscious experiences, and much more. All 
these debates are healthy and normal. The most impor¬ 
tant advance is that we now have a number of reliable 
methods for studying both conscious and unconscious 
brain events (Figure 1.30). We can contrast brain acti¬ 
vation patterns for imagined versus actual events, as 
well as for stimuli that are not consciously perceived 
(see Table 1.3). 

The most important point is that, from a scientific 
point of view, conscious cognition is much like working 
memory or attention. It is a construct inferred from behav¬ 
ioral observations (Mandler, 2003). Fortunately, it also 
corresponds usually to our own personal experience. 
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FIGURE 1.30 Unconscious versus conscious vision. A recent wave of experiments compares conscious and 
unconscious phenomena in the same study. For example, a method called visual backward masking allows us to com¬ 
pare fMRI activity in the same person for conscious and unconscious visual words. This figure from Dehaene et al., 
2001 shows that unconscious words trigger local activity in visual cortex, but matched conscious words also evoke 
strong forward activity in the cortex, in the parietal and frontal lobes. In both experimental conditions the same phys¬ 
ical stimulus reaches the retina, so that the difference must be higher up in the brain. This pattern of results has now 
been found a number of times. Evidence like this has convinced a number of scientists that conscious cognition can 
be studied with the proper kinds of experimental designs. Source : Baars et al., 2003. 


In the study of perception, for example, you can report 
the type-face of these words , or even your memory of 
this morning's breakfast. This topic has been emerging 
in a number of fields within cognitive neuroscience 
such as memory, vision, and lesion studies, as we will 
see in later chapters. 

5*2 History hasn’t stopped 

Practically all the historical debates are still alive today, 
often appearing in a new guise (see Table 1.4). The 
practical applications of cognitive neuroscience con¬ 
tinue to grow. Modern neurology makes extensive use 
of brain imaging and cognitive techniques to assess 
subtle kinds of damage that are not visible in scans. 
New research has shown unsuspected deficits, for 


TABLE 1.4 Ongoing debates today 

Local versus widespread functions in the brain 
The neuron doctrine 
The question of consciousness 
Unconscious inferences in vision 
Capacity limits in the brain 

Short-term and long-term memory: are they separate? 

The biological bases of emotions 

Nature versus nurture, genes versus environment 


example, such as emotional blindness and the inabil¬ 
ity to use certain nouns and verbs. These subtle signs 
can point to serious brain damage that is not otherwise 
detectable. 
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6*0 SUMMARY 

In this first chapter, we explored some of the ques¬ 
tions we try to understand in cognitive neuroscience. 
They encompass some ancient issues such as the mind 
versus the body, as well as some that arise from very 
recent experiments. The scientific exploration of the 
relationship between the mind and the brain is a cen¬ 
tral issue addressed by this text. The advent of imag¬ 
ing techniques has brought many of these questions 
back to life. 

Cognitive neuroscience combines psychology, 
neuroscience, and biology. The historical roots of the 
discipline go back to some of the earliest writings in 
Eastern and Western traditions. Questions that were 
out of bounds a few years ago are now being explored. 
For students, new careers are opening up. Practical 
applications are emerging in education, medicine, and 


psychotherapy. Major challenges remain, but there is a 
sense of excitement in the air. New findings are com¬ 
ing faster than ever before. 

The goal of this chapter has been to give you an 
overview of the combined study of mind and brain. 
We now have brain imaging studies of vision and 
hearing, learning and memory, conscious and uncon¬ 
scious processes, visual imagery and inner speech. 
This is an extraordinary time in science. Most imag¬ 
ing experiments use standard cognitive tasks, so there 
is a growing integration of the behavioral and brain 
evidence. The implications for brain functioning in 
health and disease are immense. 

The field of cognitive neuroscience aims to make 
sense of the flow of new evidence. This chapter pre¬ 
sented a first broad overview. Our goal is to show you 
how mind and brain evidence often fit together in a 
surprising and satisfying way. 


BOX 1.3 Adaptation or representation? 


There is another debate that continues today: should 
brains be viewed as adaptive or representational? This 
debate concerns how we think about cognitive processes 
and the way in which to map them onto brain systems. 
Here is some background. 

As you read this sentence, you are engaging in a 
symbolic interpretation of a string of black squiggles on 
a colored background. Linguists and psychologists think 
about language in terms of symbols: phonemes, let¬ 
ters, morphemes, words, phrases, and sentences. Those 
symbolic units are combined by the regularities of gram¬ 
mar and usage. Educated speakers of English under¬ 
stand about 100,000 words, often with more than one 
meaning - roughly the number of words in a good 
dictionary. At a higher level, we think in terms of sen¬ 
tences that encode abstract concepts and propositions, 
and these can be combined into even larger-scale dis¬ 
course structures, like expository paragraphs, conver¬ 
sations, and stories. Even the expressive function of 
language - the emotional singsong of a phrase, or a 
verbal statement of anger or love - can be described 
symbolically: human languages have many words for 
emotions. In sum, we are remarkably skilled in the use 
of symbols. They are essential in cultural developments 
like science, mathematics, and logic, but also in literature, 
music, and the visual arts. Even computer programs are 


symbolic systems that perform useful work. Symbols are 
all around us, and it is therefore natural to think about 
the brain as a symbolic processor. 

Yet the physical brain only contains neurons, connected 
to each other in a variety of ways. Should we think about 
the mind-brain as a symbol processor, or as a hyper¬ 
complex web of neurons? Each approach has pros and 
cons. Symbols allow us to explore questions of meaning 
and purpose fairly easily. Symbolic flow diagrams, like 
the 'functional framework' of Chapter 2, help to clarify 
questions that can be tested in the laboratory. 

On the other hand, artificial neuronal network models 
(ANNs) can simulate simple learning tasks better than 
symbolic systems (see Chapter 2). We therefore have a 
range of tools - flow diagrams, artificial neural networks, 
semantic networks, tree diagrams, and models - that 
combine symbols and neurons. In fact, if you look closely, 
even neural networks have nodes and connections with 
symbolic names. 

Scientists are constantly debating these questions. 
Cognitive neuroscience is a discipline in ferment, with 
waves of new evidence and theory constantly challeng¬ 
ing our ability to understand. It is an exciting and fun 
enterprise, and we hope that in the coming chapters you 
will catch a glimpse of what keeps us fascinated with a 
flow of new insights into human minds and brains. 
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BOX 1.4 How to study in this course 


We highly recommend drawing and coloring as a way of 
learning the geography of the brain. As you know, learn¬ 
ing requires active involvement. That is especially true in 
a course like this. There is no substitute for careful read¬ 
ing, thinking, asking questions, and exploring answers. It 
can be helpful to study with another person who is also 
focused on learning. But because the brain is a 'hyper¬ 
complex surface', drawing is especially valuable. Most 
chapters in this book will present drawing and coloring 
exercises at the end. 

The traditional way to learn the brain is by dissecting 
sheep and frogs. That is effective because the student is 
constantly interacting with the physical brain. We will 
ask you to study this text and its graphics just as if it were 
a living brain, but without having to harm any creature. 
Our figures are designed to show the many levels of anat¬ 
omy and to illustrate the most important experiments. 

Because the brain is a vast, three-dimensional structure 
squeezed over millions of years of evolution into a very 
small space, learning its ins and outs is much like getting 
to know a city, to drive or walk around it, to memorize 
the major landmarks, and to visit some of the back alleys 
and individual houses. This book will be your tour guide, 
pointing out landmarks and the customs of the inhabitants. 

For major learning points in the text there will be a 
demonstration, a graphic image, or experimental evi¬ 
dence. In the coming chapters we will see case histories 
of people who have suffered brain injuries, and explore 
some of the limits of normal brains as well. 

Finally, it is important to break down the technical 
terms into their parts. Anatomical studies of the brain 
began four centuries ago, when the international language 
of science was Latin. Many brain terms are therefore 
still compounds of Latin and Greek. For that reason it 
is important to recognize cognates - words that are simi¬ 
lar to English. For example, in this chapter we saw the 
cortex from one side (the lateral view), from the midline 
(the medial view), the bottom ( inferior view) and the top 
(superior view). Those four words - lateral, medial, inferior 


and superior - have obvious English analogs. Fortunately, 
much of our English vocabulary comes from Latin-based 
languages, so that you will often be able to spot connec¬ 
tions with familiar words. 

It will help you to do elaborative learning, rather than 
just repeating new words to yourself. The top of the brain 
is often called the dorsal part, because dorsum means 
'back' in Latin, and if you look at dogs or cats walking, 
the top of their brain is an extension of their backs. As we 
just mentioned, the side is called lateral; therefore struc¬ 
tures that are on top and to the side are called dorsola¬ 
teral. We will explain those subunits as each new word is 
introduced. 

Anatomy students often use memory techniques. You 
may want to explore them to see which ones work best 
for you. Medical students traditionally draw the brain 
and color its parts. But any kind of active association will 
help - rhyming the words, making up visual images, or 
thinking of analogies. You might visualize the brain as an 
automobile, and associate different brain structures with 
the tires or the hood. Don't worry if the associations seem 
silly, so long as you get a clear mental connection between 
what you are learning and what you already know. Just 
as there are no bad questions, there are no silly memory 
techniques. Just ones that work for you, or don't. 

Finally, self-testing is an essential ingredient of learning, 
just as it is for playing an instrument. Otherwise we tend 
to overestimate what we know. Remember that just being 
able to recognize a word does not mean that you'll be able 
to recall it when you need it. If you expect a recall test, it 
helps to practice recall. 

It may help you to browse the web for different ways of 
looking at cognitive experiments and brain images. If you 
feel ambitious, the National Library of Medicine has a free 
web database with tens of millions of scientific abstracts. 
PubMed will answer all kinds of scientific and medical 
questions (search for Entrez PubMed). There are outstand¬ 
ing brain anatomy sites, and many excellent websites that 
demonstrate basic cognitive and perceptual phenomena. 


7*0 END OF CHAPTER EXERCISES 
7*1 Study questions 

1 Name three small-scale spatial events in the 
brain, with their order of magnitude. 

2 Name three small-scale temporal events in the 
brain, with their order of magnitude. 

3 What was the dominant viewpoint about the 
nature of psychology in the 19th century? In the 
early 20th? 

4 What is a major difference between behavioral 
and cognitive psychology? 


5 What influence did physiologists like Pavlov 
have on psychology? 

6 Explain some ways in which psychology and 
brain science interact. 

7 What are some difficulties in studying brain 
damage scientifically? 

8 What is the relationship between the 'mental' 
point of view and 'physical' perspective? What 
is philosophical naturalism? 

9 In everyday life, are you aware of using inner 
speech? Visual imagery? If so, in what situations? 
If not, what everyday situations might show that 
kind of process? (Note that there are considerable 


( Continued ) 
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individual differences in visual imagery; not eve¬ 
rybody reports having spontaneous images.) 


7.2 Drawing exercise 

We highly recommend drawing and coloring to 
help you remember the physical layout of the brain. 

Here is a model brain for your use. It is very 
helpful to draw or color the following areas. Many 
people find it useful to make a habit of coloring 
brain parts in consistent colors, to help to associate 
brain regions by color. All kinds of memory aids 
can help. 

1 The major lobes (seen from the outside view, 
both right and left). 

2 What are the names of the major lobes? 

3 Where is Broca's area? Wernicke's? What are 
their traditional functions? 

4 What anatomical structure connects these two 
areas? 

5 Draw the brain from the left side, labeling the 
major lobes. 

6 Which area is associated with vision? With 
speech? With executive control? 








It seems that the human mind has first to construct forms independently before we can find them 
in things. . . . Knowledge cannot spring from experience alone, but only from a comparison of 
the inventions of the intellect with observed fact. 


Albert Einstein (1949) 
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This functional framework, used throughout this book, combines two classical models of cognition (Baddeley 
and Hitch, 1974; Atkinson and Shiffrin, 1968). Two yellow arrows symbolize voluntary (top-down) and sponta¬ 
neous (bottom-up) attention. Long-term memories, knowledge, and skills are shown in the row of gray boxes at 
the bottom. In Chapter 8 we add "conscious cognition' for reportable brain events, such as your conscious per¬ 
ception of the page in front of you (Baddeley, 2002). While the diagram is only an outline, it has the advantage of 
simplicity and wide acceptance. It should be taken as a first sketch of mind and brain functions. A recent version 
of Baddeley's Working Memory model is shown in the center. 
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1.0 INTRODUCTION 

In this chapter, we introduce a framework that will help 
to organize a good deal of cognitive neuroscience. While 
not all of the details are settled, it combines a large body 
of brain and psychological evidence into a single dia¬ 
gram. Because cognitive neuroscience can be compli¬ 
cated, an organizing framework will help you learn. 

The figure on page 32 shows our framework sche¬ 
matically, and Figure 2.1 shows some associated 


regions of the cortex. As we will see, there is both 
behavioral and brain evidence for all the compo¬ 
nents. But we will hold this organization lightly, as 
just one way to understand the evidence. When there 
are debates about how to interpret the facts, we will 
explore their pros and cons. 

In this chapter, we will briefly touch on each aspect 
of the functional framework. Each 'box' is expanded 
in later chapters. 

The left side of the figure on page 32 begins with the 
sensory systems. The senses all begin from receptor 
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FIGURE 2.1 Some major functions of the human cortex. The lateral (side) and medial (midline) 
views of cortex are colored to show major functional regions. The colored regions should be memorized 
to make it easier to understand the rest of the book. On the right of both figures are the sensory halves 
of cortex, the posterior cortex, including the occipital cortex for vision and part of the upper temporal 
cortex for hearing. The body senses are represented just behind the central sulcus (light green). On the 
left side of the central sulcus are motor regions (light purple), and in front of the motor cortex, the pre¬ 
frontal cortex for executive functions. Thus we can conveniently divide the cortex into input regions in 
the posterior half and output regions in the front half, at least as a first approximation. (This figure does 
not show subcortical regions or input-output pathways.) (See Chapter 5.) Source : Drawn by Shawn Fu. 


surfaces containing millions of receptors, like the retina at 
the back of the eye. All sensory pathways terminate in 
the rear half of the cortex, as shown in Figure 2.1. Each 
of the classical senses is believed to have a brief storage 
ability called a sensory buffer. Vision and audition have 
been studied for more than two centuries and a great deal 
is known about them. The body senses, like touch, pain 
and proprioception, are also well understood. However, 
even though the smell and touch senses were the earliest 
to evolve, they are not as well understood as the others. 
New sensory abilities are still being discovered, including 
those involved in circadian rhythms, digestion, and even 
for sexual and reproductive functions. 

Sensory activities are flexibly enhanced by selective 
attention, shown by the two yellow arrows in the fig¬ 
ure on page 32. As the diagram shows, attention has a 
'bottom-up' component, to reflect the times when our 
sensory experience is captured by a flash or a bang, or 
more subtly by the sight of someone's face. We can also 
pay attention to a range of events voluntarily in a 'top- 
down' fashion. That is how we normally call each other's 


attention to something interesting. While there continues 
to be debate about brain regions involved in selective 
attention, recent evidence shows that cortical regions for 
visual attention show marked overlap with eye move¬ 
ment control (see Chapter 8). 

While this diagram has no symbol for conscious 
experience, we will see in Chapter 8 that there is an 
intimate relationship between selective attention and 
perceptual consciousness. 

2*0 CLASSICAL WORKING MEMORY 

In the middle column of boxes in the figure on page 32, 
we have the classical components of working memory 
(Baddeley and Hitch, 1974; Baddeley, 2002). These com¬ 
ponents have been studied in great depth over three 
decades. 

Starting from the top, working memory includes the 
central executive, which is the subject of much current 
research. The central executive was first studied in learning 
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tasks. However, as the frontal lobes have become 
better understood, executive functions have expanded to 
include supervisory control over all voluntary activities 
(e.g. Luria, 1976, Goldberg, 2001; see Chapters 8 and 11). 
It is often described by the metaphor of a chief executive 
officer of an organization. For example, Mateer et ah write: 

Imagine the role of the executive of a large company, 
who has overriding control over the company's actions. 
This person sets goals, plans and organizes company 
activity to meet those goals, and decides when to start 
to do something, when to stop doing it, when to do 
something else, and when to do nothing at all. 

The executive is future directed and goal oriented 
and, to be effective, must be flexible and adaptive. At 
a basic level, this is what the prefrontal cortex does for 
humans. (Mateer et ah, 2005; italics added) 

The word 'prefrontal' means the forward part of 
the frontal lobes. The prefrontal executive regions 
are located in front of the purple motor regions of the 
brain (see Figure 2.1). They are marked in light beige 
and yellow colors, on both the outside and inside views 
of the hemispheres. Notice that Broca's area, first dis¬ 
covered in the 19th century, is technically a part of the 
prefrontal cortex. However, the light purple motor 
regions (on both sides of each hemisphere) also have 
some executive functions, though typically more local 
ones compared to the classical prefrontal cortex. These 
boundaries are not absolute, but they are convenient as 
a first approximation. Later chapters explore executive 
functions in more detail (see Chapter 12). 

Just below the central executive box in the figure on 
page 32, the diagram shows a working storage element. 
In the brain, working storage is believed to involve the 
medial temporal cortex and prefrontal regions. Working 
storage is dynamic - i.e. it involves active populations 
of neurons called cell assemblies , which can 'crystallize' 
into permanent memories. Because working storage is 
dynamic, it is more vulnerable to disruption than are 
permanent memories. 

2*1 The ‘inner senses’ 

Another step down in the diagram are two of the 'inner 
senses', labeled verbal rehearsal and the visuospatial sketch¬ 
pad. Notice that they interact constantly with the gray 
boxes along the bottom, the long-term stores. Verbal 
rehearsal is now thought to be another term for inner 
speech, the fact that human beings seem to spend most 
of their waking hours talking to themselves (Luria, 1976; 
Morin, 1993). Inner speech is not just for rehearsing 
and memorizing information; rather, it keeps a running 


commentary on our 'current concerns', while the vocal 
tract is inhibited, so that we do not express our inner 
speech out loud (Singer, 1993). Because it involves the 
sophisticated human language capacity, inner speech is 
closely tied to the linguistic and semantic component of the 
long-term stores, shown at the bottom of the diagram. 

It is easy to demonstrate basic features of verbal 
rehearsal by means of immediate memory tasks, like 
trying to remember a telephone number or a shop¬ 
ping list. In the brain, Broca's and Wernicke's areas are 
involved in inner speech, as you might expect from 
Chapter 1 (see the bright yellow and orange regions 
in Figure 2.1). However, language-related areas extend 
far beyond the traditional Broca-Wernicke cortex. As 
we will see, semantics, the representation of abstract 
concepts, also engages temporal and frontal lobes. 

The visuospatial sketchpad refers to our ability tem¬ 
porarily to hold visual and spatial information, such as 
the location of a parked car, or the route from home to a 
grocery store. Visual imagery is easy to demonstrate, for 
example by asking people to visualize their front door, 
and then asking them on which side the doorknob is 
located. Even people who do not consider themselves to 
be vivid imagers usually report 'seeing' their doorknob 
on the right or left side of their imagined front door. 
Kosslyn and others have found many brain and behav¬ 
ioral similarities between visual imagery and visual per¬ 
ception (e.g. Kozhevnikov et ah, 2005). In general, these 
authors believe that visual imagery makes use of a sub¬ 
set of visual perception areas. Thus the blue areas of the 
brain figures might participate in visual imagery. 

However, the visuospatial sketchpad also involves 
more abstract and cross-modal (involving more than 
one sense or 'mode') spatial information. For example, 
we can close our eyes and identify objects by touch, 
even though we may never have touched them before. 
There must therefore be cross-modal transfer between 
vision and touch. Such cross-modal flow of informa¬ 
tion is associated with parietal cortex (shown in green 
in Figure 2.1), sometimes called the 'Where stream' as 
opposed to the 'What stream' of the lower visual areas. 
The auditory sense also has a spatial component - we 
can locate sounds quite accurately with our eyes closed, 
especially if they have fast, high-pitched transients, like 
the chirps of a sparrow. Thus, all the sensory systems 
begin as domain-specific visual, auditory, or touch per¬ 
ception, but are quickly interpreted as part of a com¬ 
mon multimodal space that surrounds our bodies. 

The visuospatial sketchpad and the verbal rehearsal 
component of the functional framework involve men¬ 
tal capacities that we use to remember new words, 
new faces, spatial information, such as whether to turn 



36 


2. A FRAMEWORK 


right at a specific street corner, and so on. These every¬ 
day capacities have an evolutionary basis in language 
and the spatial senses. It seems likely that there are also 
'inner senses' involved with smell and taste, body sensa¬ 
tions, pain and pleasure, and the like. It has been shown, 
for example, that expected pain activates brain regions 
that overlap with those that become active for real pain. 
However, it is the verbal and visuospatial 'inner senses' 
that have been studied the most. We will focus on them. 

Long-term stores are represented by the horizontal 
row of boxes along the bottom of the figure on page 32. 
These are the brain stores for autobiographical epi¬ 
sodes, various kinds of knowledge, and practiced 
expertise. Once these memory types are stored, they 
are not conscious. However, they interact constantly 
with active functions. 

All parts of the system work with the others - and 
they may occasionally compete against some of them 
as well. For example, visual and auditory stimuli tend 
to compete if they cannot be integrated into a unified 
experience. A good example is a movie in which the 
sound and visual tracks are out of sync. When they 
are desynchronized by more than about one-tenth of 
a second, they tend to interfere with each other. When 
they are synchronized, they tend to unite perceptu¬ 
ally (McGurk and MacDonald, 1976). We perceive the 
speech as coming from the person whose mouth is 
moving in just the right way at the same moment. 


2.2 Output functions 

On the right hand side of our diagram are the output 
components. These include the central executive, action 
planning , and motor output. The best example here is 
voluntary motor functions, notably the ones that con¬ 
trol our skeletal muscles - the muscles of the torso, 
head, and face. We have no voluntary control over 
the smooth muscle systems of digestion and blood 
flow, or over the vital hormonal system. Because these 
physiological systems have long been known to be 
autonomous from voluntary control, they were called 
the autonomic nervous system. The autonomic nervous 
system is anatomically separate from the voluntary 
motor system, which is under frontal control. 

Humans also have voluntary control over some men¬ 
tal functions. A good example is verbal rehearsal and 
mental rotation of a visual image. If you can visualize a 
kitchen chair, you may be able to rotate it upside-down 
mentally. This kind of imagery task activates motor, 
spatial (parietal) and visual cortex. Because it is volun¬ 
tary, it is presumably controlled by prefrontal regions. 


To see how the functional framework can be used, 
we will consider a tragic type of brain damage, the case 
of a man who lost his ability to translate his moment- 
to-moment experiences into lasting memories. 


23 Only a fleeting moment. . . 

One day in 1985, a rising young musician in Britain 
realized that he could not remember his wife's name. 
That same evening, Clive Wearing tried to remember 
the names of his two children and failed (Figure 2.2). 
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FIGURE 2.2 Clive Wearing (with his wife Deborah Wearing) 
on the cover of Deborah Wearing's book. After losing both hippo¬ 
campi (plus some damage to frontal regions), Clive Wearing was still 
able to play piano and conduct musical pieces that he knew before 
the injury. However, he could not learn new episodic (conscious) 
events. Wearing could retain conscious experiences for perhaps 
ten or twenty seconds, suggesting that aspects of his immediate 
memory were intact. However, he was catastrophically impaired 
for episodic learning, i.e. for transferring conscious information into 
long-term episodic memory. Source: Wearing, 2005. 
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Deborah and Clive Wearing had been married shortly 
before the onset of his condition, but he could not 
recollect the wedding. His condition, which was per¬ 
manent, came on without warning after two days of 
severe headaches and fever. That was long enough for 
a viral infection to destroy regions in Clive Wearing's 
brain that are needed for new memories to form. 
Wearing was stricken with chronic, dense amnesia of 
unusual severity, including both episodic memory loss - 
he could not remember any past experiences - as 
well as a partial loss of semantic memory - an inabil¬ 
ity to understand some domains of meaning (Wilson 
et al., 1995). Most crucially, he was unable to learn 
new information. His life, as his wife Deborah later 
said, was suddenly narrowed to 'a single, blinkered 
moment', without past or future. 

Clive Wearing has been the subject of twenty years 
of news stories, studies, and television documentaries. 
His case is particularly dramatic because he seems 
unchanged as a person. He is fully conscious of the 
immediate world around him, can read and write, and 
can carry on a conversation in the present moment. 
Wearing can even conduct his small chorus in a classi¬ 
cal music piece, provided he already knows the music. 
He is an emotionally intense person, perhaps even 
more than before his injury, particularly in expressing 
his feelings for his wife Deborah. 

Clive Wearing lives in an eternal present. For the first 
eight years, he spent every day in his hospital room 
writing in his diary, trying to recapture what he called 
his consciousness. Every few minutes he wrote down 
the time of day, followed by the exclamation, 'I am 
now conscious for the first time!!' A few minutes later. 


he would cross out the previous entry, believing that he 
had not really been conscious at all, because he could 
not recall that moment. The same routine was repeated 
thousands of times, filling numerous diaries. But when 
his wife or friends came to visit, he greeted them as if 
he had never seen them before: they were familiar, but 
not identifiable. If they left for a few moments, Clive 
Wearing could no longer remember their visit. Even 
today, whenever Clive Wearing sees his wife Deborah 
he believes he has not seen her for a long time. Read 
more about the story of Clive and Deborah in Deborah's 
own words (Wearing, 2005). 

We know more about Clive Wearing's personal life 
than about others with similar damage. However, by 
far the most scientific studies have been conducted 
with a patient we know only as HM, who was first 
studied by Brenda Milner and Herbert Scoville 
(Scoville and Milner, 1957). In the 1950s, there were 
few drug treatments for epileptic seizures. A treat¬ 
ment of last resort for severe, unbeatable epilepsy was 
the surgical removal of part of the temporal lobe. In 
the case of HM, the two hippocampi and some sur¬ 
rounding regions in the middle temporal lobes were 
removed on both sides (Figure 2.3). Careful studies 
over decades showed that HM was unable to store 
new autobiographical episodes - defined as memo¬ 
ries of his conscious life experiences. However, HM 
was able to learn new sensorimotor skills, called pro¬ 
cedural memories. Like Clive Wearing, HM's ability 
to understand the meaning of language and of basic 
events was largely spared. Thus, his semantic memory - 
his ability to understand the meaning of things - was 
not seriously impaired. 



FIGURE 2.3 HM's hippocampal damage. Left: a coronal (i.e. crown-shaped) view of HM's brain, a ver¬ 
tical section from ear to ear. Notice how it compares to the normal brain section on the right. The two arrows 
on the right side show where the hippocampi are located. In HM's brain we can only see black areas where 
surgeons removed the extended hippocampal regions. Those cavities have filled with fluid, and appear dark 
on the brain scan. The two hippocampi are difficult to visualize in these cross-sections, since they are looped 
and hidden inside of each temporal lobe. (See Figure 2.4.) Source : Hodges and Graham, 2001. 
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The idea of basic differences between autobio¬ 
graphical (episodic), procedural, and semantic memory 
emerged over many years of study, including human 
patients, animal lesions, and people with healthy brains. 
In addition to these memory types, our brains have large 
perceptual memory capacities, long-lasting changes in our 
ability to perceive the world. For example, learning to hear 
musical instruments in a new song may involve new 
perceptual memory capacities. As children grow, their 
ability to perceive the sounds of speech, the constancy of 
visual objects, to see faces and identify voices, all become 
part of their permanent perceptual memories. In the case 
of Clive Wearing, it seems that his trained capacity to 
perceive and enjoy music is not impaired. (Experienced 
musicians can perceive more aspects of a musical piece 
than novices; i.e. their perceptual memories for music 
are highly developed.) 

There are other long-term capacities. Humans 
have a vast amount of knowledge about their native 
language, their culture and the surrounding world. 
Educated speakers of English can understand some 
100,000 words, and each word involves a network 
of associated knowledge. We have expert skills in 
processing grammar, discourse, and the interpersonal 
world. Most of this knowledge is unconscious at any 
given time (Bargh et ah, 2006). As far as we know, 
Clive Wearing's linguistic and semantic knowledge is 
unchanged, even with his severe brain damage. 

Humans live in a rich visual world that we know a lot 
about. For example, we know that an egg can fall from 
a table and break, but a table does not usually fall from 
an egg and break. The oddity of the second idea reflects 
our vast amount of unconscious knowledge about the 
visual world. We have no reason to believe that Clive 
Wearing's and HM's visual knowledge is impaired. 

Finally, declarative knowledge is commonly defined 
as our ability to recall facts and beliefs. It is the things 
we learn in school, which can be 'declared' as propo¬ 
sitions. 'China is a large country', 'Whales are marine 
mammals', and so on. Again, there is no indication that 
hippocampal damage impairs previously established 
declarative knowledge. It seems that existing knowl¬ 
edge and memory is largely spared with hippocampal 
patients like HM and Clive Wearing. And yet, accord¬ 
ing to Deborah Wearing, her husband's life was devas¬ 
tated. His greatest loss was in his capacity to store and 
retrieve his everyday experiences from memory. 

While debate continues about the exact role of the 
hippocampus itself, there is no doubt about the signifi¬ 
cance of the hippocampal region, located in the medial 
temporal lobe (MTL) (Figures 2.4 and 2.5). For that 
reason, it has become common to refer to the 'medial 



FIGURE 2.4 The two hippocampi. A see-through image of the 
cortex with the hippocampi nestled inside the temporal lobes. The 
red bulbs near the tips are the amygdalas, which play a fundamen¬ 
tal role in emotion. Surrounding areas of the medial temporal lobe 
(MTL) also play important roles in memory. (See Figure 2.5.) 



FIGURE 2.5 A bottom view of the brain shows regions 
removed by surgery in HM. Because more than just the hippocampi 
are believed to be needed for immediate memory storage, it is now 
usual to refer to the entire medial temporal lobe (MTL), which is 
marked. The term "medial' refers to the midline of the brain, run¬ 
ning from front to back. 
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temporal lobe' or the 'hippocampal complex' as a 
whole. Notice that the hippocampi are complex spatial 
structures, a double loop inside each of the temporal 
lobes but joining across the midline of the brain. 

It is easiest to see the medial temporal lobe from the 
bottom of the brain, which is another important view¬ 
point of the brain to be able to recognize. The brain is 
a massively complex organ, evolved over hundreds of 
millions of years, but packed into a very small space. 
For that reason, it is often important to recognize a 
structure like the hippocampal region from two or 
three points of view (Figure 2.5). 

HM is one of the classic patient cases in the history 
of science, much like Broca's speech-impaired patient 
in the 1860s. He has been the subject of more than a 
hundred scientific publications. While HM and oth¬ 
ers have been enormously important scientifically, the 
case of Clive Wearing gives us a richer human context. 
Deborah Wearing has worked for many years to com¬ 
municate the nature of hippocampal damage to the 
larger world (Wearing, 2005). Through their personal 
tragedies, such patients teach us crucial facts about 
mind and brain. 


The two hippocampi - one in each hemisphere - are 
elongated loops, nestled in the middle of each tempo¬ 
ral lobe. The hippocampi and their neighboring struc¬ 
tures in the medial temporal lobe cannot be seen from 
the outside, but they are crucial to our ability to store, 
work with and retrieve experiences from memory. 

2*4 Understanding Clive Wearing in the 
functional framework 

HM and Clive Wearing have lost the ability to move 
information from their everyday experiences into 
memory. One hypothesis about this kind of dam¬ 
age is shown in our framework diagram (Figure 2.6). 
First, consider all the functions that are not impaired 
in hippocampal damage in both hemispheres. Clive 
Wearing seems to be quite healthy in his sensory func¬ 
tions, including his ability to appreciate music; his 
conscious experiences and executive control seem nor¬ 
mal; so does motor control (though Wearing has some 
motor difficulties that seem unrelated to hippocam¬ 
pal damage); as we pointed out, some types of long¬ 
term memory seem to be unimpaired, except for his 


A functional framework 


Sensory 

input 


Bottom-up 



FIGURE 2.6 How medial temporal lobe (MTL) damage can be viewed in the functional framework. Notice 
that most cognitive functions are spared in classic cases of medial temporal lobe damage in both hemispheres. 
However, these patients have lost the ability to encode and retrieve conscious experiences - to transfer the 
present moment to lasting memories and recall them again. 
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critical inability to store and retrieve his autobiograph¬ 
ical experiences; and even his immediate memory 
persists for perhaps ten seconds. Wearing speaks and 
understands language at a high level of proficiency 
In most tests of intelligence Wearing may score well, 
except where learning and autobiographical recall are 
concerned. 

Clive Wearing is intensely upset about his condi¬ 
tion, showing that he has an understanding that there 
is something profoundly wrong. His personal relation¬ 
ships with other people are also intensely emotional. It 
is Clive Wearing's very normality that makes his con¬ 
dition both remarkable and extremely frustrating. 

Clive Wearing's crucial loss, therefore, is not in most 
of the boxes of the functional diagram. It seems that 
his deficit is primarily limited to the transfer between 
immediate memory and long-term memory, in both 


directions: between encoding conscious experiences 
and retrieving them (Figure 2.6). 

2*5 The importance of immediate memory 

Immediate memory is needed even for the simplest activ¬ 
ities. If you cannot remember the beginning of this 
sentence you cannot understand its ending. Because 
a sentence takes several seconds to read, you must be 
holding information for that length of time. Similarly, 
if your brain cannot store information from one visual 
fixation while scanning the rest of your visual field, 
you cannot put together the separate elements of the 
scene in front of your eyes (Figure 2.7). Finally, if you 
need to eat but you cannot keep your goal in mind long 
enough do something about it, you may end up going 
hungry. Research over the last fifty years supports the 



FIGURE 2.7 Immediate memory is needed to integrate small foveal fixations into a single conscious 
scene. Immediate memory is needed for virtually all cognitive functions. Long eye movements (saccades) 
jump from one point to another in a visual scene, stopping for only a fraction of a second in the most impor¬ 
tant parts of the field. At each fixation point the high-resolution part of the retina (the fovea) only picks up a 
small patch of information, shown by this classic picture from Yarbus (1967; modified by Mark Dubin, used 
here with permission). The pattern of saccadic movements and momentary fixations are shown by lines super¬ 
imposed on the girl's face, below. It is believed that the brain stores each foveal snapshot, plans the next sac- 
cade, and integrates all the small 'snapshots' into a single coherent visual representation. For that reason an 
immediate visual buffer memory is needed to integrate many fragmentary records into a coherent conscious 
scene. Source: Adapted from M. Dubin, with permission. 
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idea that all sensory, motor and cognitive functions 
require some immediate memory component 1 . 

Clive Wearing and HM have taught us a great deal. 
The hippocampal neighborhood continues to be an 
active topic of research. Because the medial temporal 
lobe is part of the ancient olfactory brain, which con¬ 
stitutes a large part of the early reptilian and mamma¬ 
lian brain, it has many different functions. The medial 
temporal lobe is also an area of great convergence 
between different sense modalities. 

Long-term stores may be located in many places, 
as we will see. These may include the entire cortex, 
and even subcortical structures like the basal ganglia 
and cerebellum (see below). But almost everything 
we can see in Figure 2.1 is the evolutionary more 
recent cortex, called the neocortex (i.e. the new cortex, 
as opposed to the ancient cortex of reptiles and early 
mammals). Neocortex expanded greatly over mam¬ 
malian evolution, accelerating during primate and 
hominid evolution of the last three million years. The 
hippocampal regions themselves are called paleocortex , 
or 'old cortex'. While neocortex has six distinct cellu¬ 
lar layers, paleocortex has four or five. In humans and 
other mammals, the hippocampal region is constantly 
interacting with the neocortex, to encode, maintain 
and retrieve memories when needed. 

The role of the hippocampal complex is one exam¬ 
ple of the constant interplay between limited-capacity 
and large-capacity abilities in the brain. 'Limited 
capacity processes' include conscious cognition, selec¬ 
tive attention, immediate memory, and voluntary 
control. 'Large-capacity functions' include long-term 
memories, highly practiced skills, and our language 
vocabulary. We will explore this theme next. 


3*0 LIMITED AND LARGE-CAPACITY 
FUNCTIONS 

Even though human brains have tens of billions of 
neurons, in some ways they have very narrow capac¬ 
ity limits. The limits of short-term memory - the 
kind we can mentally rehearse - is about 'seven plus 
or minus two' separate items, as George A. Miller 
famously described in 1956. That number seems to 
apply to many kinds of unrelated items - colors. 


numbers, short words, musical notes, steps on a rating 
scale, and so on. There are only a few conditions. One 
is that each item must be consciously noticed for only 
a brief time; if it is made consciously available many 
times, it becomes part of long-term memory. Second, 
the items must be unpredictable from previous knowl¬ 
edge. If we simply ask people to remember a rule, like 
the number series 1, 5,10,15, 20, 25 . . . (etc.), they can 
keep much more information in immediate memory 
because they only have to remember the rule. When 
we are prevented from mentally rehearsing items in 
immediate memory, the capacity of immediate mem¬ 
ory drops from about seven to less than four separate 
items (Cowan, 2001). 

Selective attention is another limited capacity function. 
It was initially studied by Donald Broadbent and others, 
using the dichotic listening paradigm (Figure 2.8). In this 
method, subjects wear headphones with two separate 
speech channels, often one into each ear. Because they 
are asked to 'shadow' one of the two channels (i.e. to 
repeat the incoming speech with minimum lag time), 
subjects can hear only one channel. It is easy to dem¬ 
onstrate this by listening to two radio news broadcasts 
at the same time, for example. Under these condi¬ 
tions people can only understand one channel, though 
they can pick up the voice quality in the unattended 
channel. 

For a gigantic brain, these capacity limits are tiny. 
Compared to a digital computer, for example, they are 
astonishingly small. But surprising limits are found 
beyond immediate memory and selective attention. 
Table 2.1 shows a dozen other phenomena that show 
similar, narrow capacity limits. 

3.1 Dual task limits 

Personal technology has made it possible for us to try 
to study, listen to our favorite music, and talk on a cell¬ 
phone at the same time. Sadly, our efficiency in multi¬ 
tasking goes down rapidly the more different things we 
try to do at the same time. We cannot generally do even 
two conscious things at a time, such as carrying on a 
complicated conversation and driving in rush hour traf¬ 
fic. If we don't need to think much about talking, we can 
drive at the same time, and vice versa, but the more con¬ 
scious involvement is needed for each task the more they 
will compete. For that reason, dual-task methods are 


1 There is also evidence for an intermediate memory capacity lasting hours or days, but not longer - to remember where we left a bicycle, what 
food to buy for dinner, how to navigate a new environment, or recent interpersonal events (Delaney et al, 2004). 
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often used to study how much of our limited capacity 
is taken up by a task. As the cognitive demands of one 
task rise, the efficiency of the second task will go down 
correspondingly Dual-task methods have therefore been 
immensely important in the development of cognitive 
neuroscience. 

How do we cope in the face of capacity limits? 
There are at least two ways. One is chunking , the abil¬ 
ity to condense a vast amount of information into a 
single, organized unit. The words of human languages 
are often chunks: imagine the amount of information 
we can access using words like 'mother', 'school', 
'love', and so on. We have many other ways of con¬ 
densing information into single chunks. In studying 
this chapter, chances are that you will condense it into 


FIGURE 2.8 Donald Broadbent and selective attention. 
Renewed interest in selective attention emerged in the 1950s with an 
influential experimental program started by Donald A. Broadbent. 
The upper image shows Broadbent's Tunnel' image of limited 
capacity functions, which dramatizes the fact that our large brains 
have narrow limits for selective attention, conscious perception, and 
voluntary control. Broadbent's selective listening task, in which two 
messages are sent simultaneously to the two ears, is a classical dual 
task method for studying limited capacity functions. Photograph 
Source: From Broadbent, with permission. 

main points. That is a good chunking strategy, because 
you can summarize thousands of words into a few 
dozen sentences. 

We can also get around capacity limits by practice. 
In general, humans are much better with practiced 
tasks than with novel ones. The brain seems to solve 
novel problems rather slowly; it makes many errors, 
and tends to sequence even things that might be done 
at the same time. Conscious access drops when skills 
become more practiced and efficient (Raaijmakers 
and Shiffrin, 1992; Chein and Schneider, 2005). Thus 
a skilled video gamer may be able to talk at the same 
time as playing the game, while a novice must concen¬ 
trate on the game. Practice is a way of lowering the 
limited capacity cost of doing a task. 
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TABLE 2.1 Limited capacity tasks 


1 Dual-input limits. Some fifty years of research shows that people cannot consciously understand two incompatible streams of information at the 
same moment. 

a Selective listening: receiving two streams of auditory input at the same time, 
b Inattentional blindness: tracking two competing visual streams on a single screen, 
c Binocular rivalry and its variants: receiving different visual input to the two eyes. 

2 Immediate memory limits, including the capacity to hold recalled memories. 

3 Ambiguous stimuli and meanings, such as the Necker cube, ambiguous words, and many other cases where input can be interpreted in more 
than one way. 

4 Competition between different features of the same object, such as the Stroop color-naming effect (see Chapter 3). 

5 Conjoined feature search: for example, searching for both color and shape in a complex display. 

6 Effortful tasks compete against each other. The more difficult two tasks are perceived to be, the more they tend to interfere. There may even be an 
upper bound on the number of effortful tasks one can accomplish per day (Muraven and Baumeister, 2000). 

7 Response competition. Two different output plans or actions tend to compete against each other (Pashler, 1989). 

8 Limits on temporal integration: 

a Attentional blink: in a rapid series of visual letters, a'blind'period may occur some 200-500 milliseconds after a target letter, 
b Change blindness: the difference between two visual scenes may go unnoticed if a brief white flash interrupts them. This may be a limit on the 
construction of coherent temporal events. 

9 Long-term memory search may be limited, as in the case of word retrieval difficulties. 

10 Conceptual incompatibility and functional fixedness. It may be difficult or impossible for people to understand an'obvious'problem from an 
unexpected point of view. 

11 Domain-specific limits. Specific input and output modalities may have local limitations. One example is the very small size of high resolution 
vision, using the fovea. Another is our difficulty in doing certain bimanual actions, such as patting one's head and rubbing one's stomach at the 
same time. 


TABLE 2.2 Very large-capacity functions 


Some brain features show massive capacity. 

1 The various kinds of long-term memory. 

a Episodic and autobiographical memory has been estimated to be 1000 000 000 bits (Landauer, 1986). Using recognition memory, one can 
retrieve old movie scenes, childhood landmarks, the faces of fellow students, and the like, dating back five decades or longer, 
b Semantic memory for facts and concepts is very large, 
c Procedural memory for highly practiced skills. 

2 The language vocabulary: educated speakers of English can recognize about 100 000 words, each of which involves a complex network of 
related ideas, sounds, and written words. 

3 The great complexity of sensory and motor processes. 

4 The vast number of neurons, connections, and functional networks in the brain. 


However, there is another side to the capacity 
story. Some cognitive capacities are very large indeed 
(Table 2.2). 


3*2 Some very large brain capacities 

At the level of the cells, a structure like the cerebral cor¬ 
tex is immense - a vast, looming starship unto itself, 
containing, by recent estimates, between 10 and 100 
billion neurons. Seen from the outside, it is an elabo¬ 
rately folded structure with many hills and valleys, 
neatly tucked into the upper half of the cranial cavity. If 
we could carefully unfold the great cortical mantle we 
would see a sheet about three feet square (lm * 1 2 3 4 ), with 


six layers, each composed of myriads of bushy neurons 
surrounded by supportive cells. This layered sandwich 
can be parsed into millions of vertical columns, so that 
we can imagine a vast six-layered array in three dimen¬ 
sions. Each layer seems to specialize in input, output, or 
internal connections in the cortex (see Chapters 3 and 5). 

Cortical neurons are connected by vast tracts of 
axonal fibers , wrapped in white sheathing cells called 
myelin. If we simply slice through cortex we see mostly 
white matter, an indication of how many connective fib¬ 
ers there are. To the naked eye, cortex looks like a dou¬ 
ble fruit covered with a thin skin of cell bodies, the gray 
matter that we see from the outside. But the white mat¬ 
ter contains miles of tiny axons that descend from the 
gray cell bodies and end up coming back to cortex on 
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the opposite side of the brain. Current estimates for the 
left-to-right fibers that cross via the corpus callosum - 
the large fiber bridge between the hemispheres - is 
on the order of 200 million. Each of these fibers sends 
an electrochemical message about ten times per second, 
making for message traffic of about 2 billion events per 
second. 

An equally large number of axon bundles loop 
beneath the cortex and come back up on the same side. 
Thus, there is an immense amount of traffic between 
the two hemispheres, and within each of them as well. 

The outermost layer of the six-layered sandwich. 
Layer I, is so densely woven horizontally that it has 
been called a feltwork, a large skin of tight webbing on 
top of the sandwich. While most long-distance com¬ 
munication in cortex seems to run through long vertical 
output fibers (axons), the top layer is so tightly intercon¬ 
nected horizontally that many brain scientists believe 
there is also a great deal of spreading activity within it. 

Cortical neurons do more than connect with other 
cortical neurons. They also project in vast elegant fiber 
bundles to the neural organs nestled tightly under the 
cortex, like small bird eggs in a nest. Among the bird 
eggs, the thalamus serves as a great traffic hub, the way 
station for almost all sensory messages going to the cor¬ 
tex, and therefore a strategic control point. An equally 
large mass of fibers goes from cortex to the basal ganglia 
and cerebellum, both necessary for normal movement. 

When we become conscious in the morning, the 
brain is globally activated, every part showing faster 
and more widely connected neural traffic. It is as if 
suddenly all the neurons jumped into their cars and 
decided to go places. As a novel or surprising event 
catches our attention, a vast electrical tidal wave rushes 
all over the brain, a few tenths of a second after the trig¬ 
gering event. This is called the 'event-related potential', 
discussed in Chapter 4 and in the Appendix. 

There is good evidence that anatomical regions in 
the brain can serve very specialized functions (e.g. Luria, 
1976; Geschwind, 1979a). A number of scientists have 
interpreted the organization of the cerebral cortex 
in terms of distributed 'unit modules' (Mountcastle, 
1978; Edelman and Mountcastle, 1978). Rozin (1976) 
viewed the evolution of intelligence as an increase in 
the ability to apply these specialized functions to life 
situations. Your ability to read, for example, is not 
something human beings are born with. Unlike heard 
and spoken speech, babies do not spontaneously learn 
to read. It takes years of training. But we can learn to 
use our visual system astonishingly well to take in 
language by eye rather than ear, and to express lan¬ 
guage by writing instead of our vocal apparatus. The 


ability to learn sign language and read in Braille tells 
the same story. Reading seems to ride on biological 
preadaptations. 

Rozin and others suggest that brain capacities tend 
to evolve as specialized adaptations. But in the human 
lifetime, he suggests, specialized functions can become 
available for new adaptive purposes. 

3.3 Why are there such narrow capacity limits? 

It would be nice to be able to do half a dozen things 
at the same time. Why do some functions seem to be 
so limited in a brain with tens of billion of neurons? It 
isn't just that we have a limited capacity to do things - 
only one mouth to speak with and two hands to hold 
things with. Capacity limits also operate in percep¬ 
tion, the input system, so the limitations of hands and 
mouth are not the only reason. Ambiguous figures, 
like the famous Necker Cube, are very limited: we 
can only perceive one interpretation of an ambiguous 
stimulus at any given moment. 

So the problem isn't just the fact that we can only 
do one thing at a time in the motor system. And it is 
not that the brain lacks sheer processing capacity - its 
ability to store and transform information is beyond 
our current ability to describe. Some scientists have 
argued that capacity limits are due to the role of con¬ 
sciousness in combining numerous components of a 
very large brain into an integrated whole (Baars, 1988, 
2002b; Edelman, 1989; Llinas and Pare, 1991). Limited 
functions are closely associated with conscious experi¬ 
ence, while very large capacity functions are generally 
unconscious (see Tables 2.1 and 2.2). However, it is not 
clear at this time why that should be so. 

3.4 Measuring working memory 

The middle column in the figure on page 32 corresponds 
to classical working memory (Baddeley and Hitch, 1974; 
Burgess and Hitch, 1999; Baddeley, 2000). Working 
memory emerged in cognitive psychology several dec¬ 
ades ago when scientists began to understand how 
important immediate memory capacities are. It provides 
a useful way of thinking about the cognitive brain. More 
than ten thousand studies have been published on the 
topic, and working memory is often used to provide 
a perspective or other functions as well - including 
mental imagery, language, inner speech, and executive 
control. The amount of psychological evidence about 
working memory is immense and, in the last ten years, 
our understanding of their brain basis has expanded 
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very rapidly. Working memory therefore offers a con¬ 
venient and widely understood way of approaching the 
complexity of the mind and brain. 

According to Cowan et al. (2005), 'Working Memory 
is the set of mental processes holding limited infor¬ 
mation in a temporarily accessible state in service of 
cognition'. This broad definition will prove useful. 
Working memory has come to stand for the many 
things we can keep temporarily accessible. Its mean¬ 
ing has broadened considerably since the concept 
was first proposed in somewhat different words by 
Atkinson and Shiffrin (1968) and by Baddeley and 
Hitch (1974). But, in a general sense, the working 
memory framework has remained quite consistent. 

Notice that Cowan's definition is broad enough to 
allow for expansion to other domains, not just verbal 
rehearsal and the visuospatial sketchpad. For example, 
we can imagine mentally humming a song. If we do 
that successfully, and we can verify it empirically, we 
could consider adding a working memory component 
of 'inner music'. Experienced musicians seem to have 
a number of ways of manipulating notes and melodies 
in their heads. Athletes can mentally practice sports, 
and their actual performance improves as a result. 
Scientists are currently considering different types of 
working memory for eye movements and emotional 
processes, perhaps even for dreams and daydreams. 
Working memory in this very general sense can use a 
number of different modalities. A great deal of brain 


and behavioral evidence supports a wider application 
of these basic ideas. 

Once sensory information is attentionally selected, it 
is often said to become available to working memory. 
By Cowan's definition (above), that means that an odor 
or a taste could be 'in a temporarily accessible state in 
service of cognition'. If you dwell on that definition for 
ten seconds or so, you are holding your thoughts in 
working memory. As you can tell, your ability to think 
about it is vulnerable to distracting events, including 
your own thoughts and feelings - and reading this sen¬ 
tence. Your ability to think about any topic is therefore 
capacity-limited. Thus, we might talk about 'thinking 
working memory', or 'conceptual working memory', 
as well as visual or verbal working memory. 

Measurement is crucial in science, and work¬ 
ing memory is typically assessed in several different 
ways. Figure 2.9 shows one way in which visual work¬ 
ing memory may be assessed, simply by presenting 
visual shapes, in a series of slides over specific time 
periods, often measured in hundreds of milliseconds. 
Any stimulus may be presented and re-tested some 
seconds later, either asking for recall or presenting the 
original stimulus for recognition. Behaviorally one can 
assess working memory by accuracy of recall or recog¬ 
nition, and by the speed of responding (reaction time). 

These simple but powerful methods allow for numer¬ 
ous variations. One popular method is delayed match to 
sample (DMTS), in which subjects are asked to respond 


Verbal memory condition 


Spatial memory condition 



FIGURE 2.9 This figure shows one way in which visual working memory may be 
assessed, simply by presenting visual shapes, in a series of slides over specific time peri¬ 
ods, often measured in hundreds of milliseconds. Any stimulus may be presented and re¬ 
tested some seconds later, either asking for recall or presenting the original stimulus for 
recognition. 
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3-back task 



2-back task 

FIGURE 2.10 A difficult continuous memory task: in the 
n-back Working Memory tasks, subjects are asked to recall the item 
presented one, two, or three slides before. This is a very demand¬ 
ing task. When n-back tasks are compared on fMRI, cortical activity 
increases markedly as n goes from 1 to 4. (See Figure 2.11) 

when they see a match with their recent memory. This 
method can be used for animals and infants as well as 
adults. 

What about brain assessment of working memory? 
Figure 2.10 shows an 'n-back task', an important var¬ 
iant of the simple WM task. N-back tasks allow us to 
vary the degree of mental workload, a variable that 
has major brain effects. The concept is fiendish but 
simple. In any series of stimuli, your job as a subject 
is to remember the last one you saw - that is fairly 
easy. But now you are asked to remember the second 
word you saw before the current slide, then the third 
before, and so on. To do this, subjects must rehearse 
the last n stimuli and not confuse the n-back stimu¬ 
lus with the others that have to be kept in working 
memory. 

This is subjectively harder and harder to do as it n 
grows from one to three or four, and the brain shows 
consistently wider activation. For example. Figure 
2.11 shows blood-oxygen changes at the bottom of the 
brain, using functional magnetic resonance imaging 
(fMRI) (see Chapter 4). The bottom of the temporal 
lobe contains many neurons specialized in visual object 
perception. In addition, it is adjacent to the medial 
temporal lobe, which contains the two hippocampi. 
Because this is a demanding executive task - the 
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FIGURE 2.11 Brain activity increases with the Working 
Memory load. Brain imaging using fMRI shows dramatic increases 
in the amount of cortex involved in the n-back task, as n rises from 0 
to 3. In the 0-back condition, subjects only need to name the visible 
slide. On these brain images, brighter colors signal higher levels of 
activity. Source: Smith and Jonides, 1997, Figure 10. 


cognitive equivalent of juggling half a dozen balls at 
the same time - we also see increased activity in pre¬ 
frontal regions. 

4*0 THE INNER AND OUTER SENSES 

There is now good evidence that endogenous (internally 
generated) events imitate the outer senses to some 
degree (Figure 2.12). To see the similarity between them 
you should recall that the posterior half of the cortex is 
mainly sensory. Even though our eyes, ears, nose and 
mouth point forward - the direction we usually move - 
all sensory pathways terminate in the back of the cortex, 
where sensory stimuli are analyzed at the highest level. 
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FIGURE 2.12 Framework diagram with sensory emphasis. The sensory systems receive input from arrays 
of receptors which transform physical energy patterns into neuronal firing. Sensory cortex in vision and the body 
senses is mapped to the stimulus array in a topographical fashion. 


Each sensory pathway reaches cortex in its own 
primary projection area. It is easiest to remember the 
primary cortices as VI (for the primary visual pro¬ 
jection area), A1 (for primary auditory), and SI (for 
primary somatosensory cortex, the body map). VI is 
an accurate map of the retinal input (Figure 2.13). It 
projects in its turn to V2, V3, and so on, each higher 
visual area representing the visual array in more and 
more abstract ways (see Chapter 6). As the visual sig¬ 
nal traffic flows into parietal cortex, it becomes inte¬ 
grated with hearing, touch, body space, and motor 
control. 


4.1 The mind’s eye, ear, and voice 

In the 4th century BCE, Aristotle suggested that visual 
images were 'faint copies' of visual sensations, i.e. he 
thought that imagery was a kind of vague internal 
perception. In the last few decades, a mounting body 
of evidence seems to show he was right. The American 
psychologist, C.W. Perky, demonstrated this elegantly 
early in the 20th century when she showed that people 


can confuse faint visual pictures with their own men¬ 
tal images (Perky, 1910). 

Psychologists have devised a number of methods to 
test visual imagery. Stephen Kosslyn has demonstrated 
that 'the mind's eye' is a surprisingly realistic figure 
of speech. The human visual field has a characteristic 
size and shape, which is easy to demonstrate. Simply 
look at an object in the room in which you are read¬ 
ing this, allowing your eyes to fixate on a point with¬ 
out moving your head. Now bring your hands in from 
the sides of your visual field until you can barely see 
them; the horizontal limits of the active visual field 
will be on the order of 120 degrees of visual arc. Do 
the same for the vertical limits, and it will turn out to 
be less than half of that. The working visual field seems 
to be a flat oval, perhaps 45 visual degrees in height 
by 120 degrees wide. 

If you now close one eye and fix the open eye on 
a single target, like a single letter in this sentence, the 
field will shrink dramatically to only a few degrees of 
visual arc, corresponding to foveal vision. The fovea 
is a small, central patch of each retina that has very 
high density of visual receptors, hence the highest 
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FIGURE 2.13 Occipital activation for visual stimulation. This brain scan shows activation in the occip¬ 
ital cortex, which is mapped point to point to the retinal light input. Source : Singh et ah, 2000. 


visual resolution. It is the keyhole-size 'sight' that we 
aim at the world to get high-resolution snapshots. The 
fovea subtends about four degrees of visual arc. 

You can measure your inner field of your 'mind's 
eye' in much the way you did with your visual field - 
naturally, by using your imaginary hands. Closing 
your eyes, move your virtual hands to the sides of 
your 'mind's eye', and write down the horizontal 
extent of your field. Now do the same in the verti¬ 
cal dimension. People generally will come up with 
somewhat less than 120 degrees of horizontal arc, and 
about 45 degrees vertical. A variety of such experi¬ 
ments shows a remarkable resemblance between the 
physical visual field and its mental double. Over the 
last several years, research has begun to reveal the rea¬ 
son for this resemblance. Kosslyn, Martha Farah and 
others have shown that visual imagery elicits activity 
in parts of the visual cortex. Ganis et al. (2004) write 
that, 'Visual imagery and visual perception draw on 
most of the same neural machinery'. 


However, there are many ways of seeing the world, 
and many ways to use one's visuospatial imagery. 
Depending upon experimental conditions, different 
patterns of activity may be found in visual cortex. By 
using conditions designed to match visual perception 
and visual imagery as much as possible, Ganis et al. 
showed that brain activity from visual imagery can 
be nearly identical to visual perception. Figure 2.14 
shows that the two activity patterns can be subtracted 
from each other, point by point, leaving few visible 
differences. This is a striking result. 

Kosslyn (1994) points out that imagery, 'is not a uni¬ 
tary ability, but consists instead of a host of specialized 
abilities'. The famous mental rotation task devised by 
Shephard and Cooper (1982), for example, seems to 
require visual, spatial, motor, and executive regions 
(Figure 2.15). Because VI has point-to-point mapping 
to the retina, Kosslyn wrote, 'It is possible that (VI) 
is activated only by tasks that require high resolution 
images'. 


PERCEPTION IMAGERY PERCEPTION - IMAGERY 



FIGURE 2.14 Visual imagery may activate parts of visual cortex. In these brain scans from Ganis et ah (2004), activity for perception and 
imagery are so similar that they can be subtracted from each other, yielding very little difference. As the right-most figure shows, these virtual 
slices were selected from the occipital and parietal region. Note that Perception - Imagery means perception minus imagery effects. The three 
images on the left are coronal cross sections cut vertically through the brain with the cerebellum visible at the bottom. On the far right we see a 
medial left-facing brain. Source: Ganis et ah, 2004. See mini-atlas at the front of this book. 
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(a) 




FIGURE 2.15 Different imagery tasks, 
(a) The classic mental rotation stimuli from 
Shephard and Cooper (1982). The subject is 
asked to report whether the two arbitrary 
shapes are the same or different. To answer the 
question, subjects mentally rotate one shape 
to see if it matches the other, (b) A classic 
Tower' task, which can be thought of as roll¬ 
ing colored balls from one pocket to another. 
How can you transform the upper picture into 
the lower? Here again, subjects appear to use 
visual imagery, but the task is quite different 
from mental rotation, or from the task shown. 
Source : Heslow, 2002. 


4.2 The imagery sketchpad may use visual 
regions of cortex 

Can you remember the first time you saw this book? If 
you can, do any visual images come to mind? Can you 
bring to mind the place and the way the book looked 
to you? Did you see it lying flat, or propped up? Was 
it right side up or upside down? People vary in the 
vividness of their spontaneous mental imagery, but 
most people can do these tasks. Where does this hap¬ 
pen in the brain? Figure 2.14 suggests similar brain 
regions are active when seeing versus imagining. 

Notice how well cognitive and brain findings con¬ 
verge in these examples. Vision involves occipital, 
temporal, and parietal cortex, and so does 'mental 
vision' or visual imagery, under carefully controlled 
conditions. 

4.3 Is inner speech like outer speech? 

Most human beings go around the world talking to 
themselves. People are often quite willing to tell us 
about their private monologue. Simply by asking them 
to write down clear internal speech as soon as it occurs, 
a body of useful evidence has been gathered. If we 
include inner speech in the inner senses, we can find sim¬ 
ilarities between inner and outer articulation of words. 

The psycholinguist Gary Dell has shown that inter¬ 
nal tongue-twisters create errors very similar to overt 
tongue-twisters (Dell and Sullivan, 2004). For exam¬ 
ple, try repeating, 'Peter piper picked a peck of pick- 
led peppers' in your inner speech, as quickly as you 
can. Do you notice any inner pronunciation errors? 
But you have no inner tongue to twist - or do you? 
Imaginary practice can be effective - which makes a 
lot of sense if we use the same brain tissue for mental 
and physical practice. 



FIGURE 2.16 Approximate location of Broca's and Wernicke's 
areas. Although 19th century physicians were not equipped with 
sophisticated brain recording instruments, their conclusions con¬ 
verge well with modern imaging studies. Source: Squire et al. f 2003. 


Given what we know about Broca's and Wernicke's 
areas from Chapter 1 (see also Figure 2.16), where 
would you predict activation for silent verbal rehearsal? 

Mentally rehearsing numbers to ourselves may 
sound like a vague kind of speech. Scientists have 
speculated for many years that we actually speak 
to ourselves silently, and there has been indirect evi¬ 
dence for that hypothesis. It has now been supported 
by functional brain imaging (see Figure 2.17). 

In later chapters, we will see things become a 
little more complicated, because there are many 
ways of speaking and listening, and many ways of 
seeing and visualizing. Yet this is the big picture: it 
does seem that the 'outer senses' have corresponding 
'inner senses', like visual imagery and internal speech 
(see Figure 2.18). 
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(c) 


FIGURE 2.17 Inner speech can be considered normal speech 
(a) with the vocal organs inhibited ('covert'). Covert speech uses the 
classical speech areas of the left hemisphere ((b) and (c)). A sum¬ 
mary brain figure showing some areas activated in an inner speech 
task. (B = Broca's area; W = Wernicke's area; M = motor cortex; 
A = auditory cortex). Source : Heslow, 2002. 


4 A Is there only one working memory? 

We have used the term 'working memory' as if it were 
a single thing, but that is a hotly debated question. As 
we will see, there is evidence for both 'domain spe¬ 
cific' and 'non-specific' temporary holding memories 
(see Chapter 8). Some researchers talk about working 
memories for concepts, for space, and for semantics. 


Overt playing 


Covert playing 




(b) 

FIGURE 2.18 An inner musician? The 'inner senses' are not 
limited to verbal and visuospatial abilities. We may have 'inner 
musicians' and 'inner athletes' as well. These brain scans compare 
overt and covert instrumental playing by amateur musicians. While 
overt play shows higher activation in motor and somatosensory 
areas, generally similar regions of the brain show activity in both 
conditions. Source : Lotze et al., 2000. 


as well as vision and speech. Current evidence favors 
a combination of the two hypotheses: there seem to be 
both domain-specific and non-specific working mem¬ 
ory capacities. 


5.0 THE CENTRAL EXECUTIVE 

As mentioned before, the prefrontal lobes play an 
important executive role in the brain. They are needed 
for voluntary control over actions. Prefrontal regions 
also support emotional processes and seem to be 
necessary to control one's own unwanted impulses. 

The neurologist Oliver Sacks writes (in Goldberg, 
2001b): 

The frontal lobes are the latest achievements of the 
nervous system; it is only in human beings (and great 
apes, to some extent) that they reach so great a devel¬ 
opment. . . . they lack the simple and easily identifi¬ 
able functions of the more primitive parts of the 
cerebral cortex, the sensory and motor areas. . . but 
they are overwhelmingly important. They are crucial 
for all higher-order purposeful behavior - identify¬ 
ing the objective, projecting the goal, forging plans to 
reach it, organizing the means by which such plans 
can be carried out, monitoring and judging the conse¬ 
quences to see that all is accomplished as intended.. .. 
Without the great development of the frontal lobes 
in the human brain (coupled with the development 
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FIGURE 2.19 Hannah Damasio's computerized reconstruc¬ 
tion of the brain damage suffered by Phineas Gage in 1848, based 
on his death mask (Damasio et al., 1994). At the time, most people 
would have died from an infection of the wound, if not from brain 
damage and blood loss. Gage was fortunate to survive and recover 
most of his abilities, but he could no longer pursue his old life goals 
or control impulsive actions. These changes were so profound as to 
signal a change in personality. Similar phenomena are observed in 
other cases of frontal lobe injury (see Chapter 12). Source : Caplan 
and Gould in Squire et al, 2003. 


of the language areas) civilization could never have 
arisen. 

Thus in the famous case of Phineas Gage (Figure 2.19) - 
a railway foreman who, while setting an explosive 
charge in 1848, had a two-foot tamping iron blown 
through his frontal lobes when the charge backfired - 
while there was preservation of Gage's intelligence 
as well as his ability to move and talk and see, there 
were other, profound changes in him. He became reck¬ 
less and improvident, impulsive, profane; he could no 
longer plan or think of the future; and for those who 
had known him before, 'he was no longer Gage'. He 
had lost himself, the most central part of his being. 
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FIGURE 2.20 The Stroop Color-naming Task reflects executive 
functions. Try to name the colors on top, and you are likely to find 
it difficult. When the words are unreadable, color naming is easier, 
faster, and more accurate (bottom half). Source : Miller and Wallis in 
Squire et al., 2003. 


and (as is the case with all patients with severe dam¬ 
age to the frontal lobes), he did not know it. 

The Stroop Color-naming Task is commonly used to 
test for frontal lobe damage. In the Stroop Task a con¬ 
flict is set up between reading a word and naming its 
color. By using printed color names we can present the 
word 'blue' in a green color, and ask people to name 
only the colors as quickly as possible (Figure 2.20). 
Since educated people are highly practiced readers, 
their automatic tendency is to read the word rather 
than name its color. The task instructions therefore 
ask them to do the opposite of what they have prac¬ 
ticed doing for many years. In people with healthy 
brains, the Stroop task sets up a conflict involving pre¬ 
frontal cortex, but it can usually be resolved. Response 
times are slowed compared to a control condition, and 
people sometimes make errors. But, in frontal lobe 
patients, the effects of the conflict is more debilitating. 
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leading to more errors, longer response times, and a 
greater sense of subjective difficulty and frustration. 
The Stroop Test is useful to probe for subtle frontal 
lobe damage that may be missed by brain scans. 

5.1 Executive effort and automaticity 

A remarkable finding from brain scanning experi¬ 
ments is that many different tasks involving executive 
effort all Tight up' two crucial regions of the frontal 
brain. On the sides of the frontal lobes, this involves 
the dorsolateral prefrontal cortex (DL-PFC) 2 . Along the 
midline of each hemisphere, an important executive 
region is the front of the cingulate cortex, the anterior 
cingulate cortex (ACC). These areas also show high 
activity in the Stroop Task, which involves a conflict 
between highly practiced skills like word reading, and 
a novel task like color naming (Duncan and Owen, 
2000; Frackowiak, 2004). 

Voluntary actions become automatic with practice 
(Shiffrin and Schneider, 1977). As they do so, we also 
tend to lose some executive control over them (e.g. 
Langer and Imber, 1979). Our loss of control over highly 
practiced and predictable habits seems to go along with 
a loss of conscious access to their details (Schneider, 
1995). In brain scans, we see a dramatic reduction of 
cortical activity when a predictable voluntary action is 
practiced to the point of automaticity. There is evidence 
that routinized voluntary actions may be taken over 
in part by subcortical regions of the brain, notably the 
basal ganglia and cerebellum. 

However, we should not make an all-or-none distinc¬ 
tion between voluntary and automatic actions. Automatic 
actions can come under voluntary control again when 
predictable aspects of the action become unpredictable, 
as when we break a leg and try to walk in our usual 
way (Sacks, 1984). Most everyday activities are a mix¬ 
ture between voluntary and automatic control. The most 
highly practiced components of our habitual actions 
tend to be automatic, while the most novel and unpre¬ 
dictable ones tend to remain under voluntary control. 


Thus, we may be able to make voluntary decisions about 
which way to walk at a new street intersection, but 
once we decide to turn right we are rarely conscious of 
each step we take. The same general point applies to 
speaking, reading, eye movement control, and much 
more. Automatic and voluntary control work hand in 
hand. 

An interesting point is the existence of dual-control 
systems in the brain. For example, we can smile vol¬ 
untarily or spontaneously (Figure 2.21). These similar 
muscle actions are triggered by different regions, the vol¬ 
untary one being more cortical, using the frontal lobes. 

Likewise, we can take a deep breath 'at will', but 
we normally breathe in a spontaneous and automatic 
rhythm. Large eye movements can also be control¬ 
led voluntarily as well as automatically. For smiling, 
breathing, and eye movements, automatic control is 
much more common than voluntary cortical control. 
As we will see, selective attention also has dual con¬ 
trol, both voluntary (executive) attention and sponta¬ 
neous attentional selection. 

Such dual control is a common strategy of the brain, 
but it does not apply to vital functions, like the control 
of heart rate. For obvious reasons, heart rate control 
is automatic; it would be disastrous to try to stop or 
start it at will. The same is true for other autonomic 
functions. Brain injuries can sometimes dissociate vol¬ 
untary and automatic control centers from each other, 
as happens in disorders that impair only voluntary 
smiles, but not spontaneous ones (see Figure 2.21). 
There are also brain lesions that work the other way: 
damage to the brain-stem can cause automatic control 
to be disabled while voluntary control is spared. 

There is a debate whether voluntary actions are 
equivalent to consciously decided ones (Shiffrin, 1995; 
Schneider, 1995). This is a difficult issue to settle. 
However, there is little disagreement that voluntary 
control and conscious cognition are strongly associated 
with each other. 

Recent evidence also suggests that the executive sys¬ 
tem corresponds to the 'self of everyday life, (Vogeley 


2 Terms like "dorsolateral prefrontal' sound intimidating, but are actually quite simple. They are much like the compass terminology 'North, 
South, East, West, North-West, etc'. 'Dorsal' means 'upper' in the human brain, lateral means 'side', while 'medial' is along the midline. 
'Inferior, superior and middle' are what you might expect. We therefore get compound names, such as dorsolateral, meaning the upper part 
of the side of the cortex, and 'ventromedial', the lower half of the midline of the brain. If that is hard to visualize, it is an excellent idea to draw 
these directional terms on some visual object, like a picture of a shoe or car. Some students prefer to think of a pair of shoes, to represent the 
left and right halves of cortex. These objects are useful because they have a clearly defined front, back, left and right side. If you then simply 
translate the Latin-based vocabulary into words like 'front, back, upper, lower, middle, and sides' you will find it easy to cope with this termi¬ 
nology. We will use a hyphen to separate the directional part from the brain part. Prefrontal cortex is abbreviated as PFC. The upper side part 
of the PFC is therefore 'DL-PFC'. When in doubt, be sure to decompose the long words into their subunits. 
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FIGURE 2.21 Selective damage to voluntary but not sponta¬ 
neous smiles. On the left, this patient cannot make a symmetrical 
smile, while on the right her smile looks quite normal. On the left, 
she is trying to smile voluntarily, while in the right photo she is smil¬ 
ing spontaneously The damage to her frontal lobe motor regions 
does not affect her (subcortical) facial expressions. Other patients 
show the opposite damage, impairing spontaneous smiles but not 
voluntary ones. The ability to show that two brain functions can be 
damaged independently of each other is called a double dissocia¬ 
tion. Source : Paxinos and Mai, 2004. 



FIGURE 2.22 Posner's model of executive attention. Selective attention is part of the working framework. It 
can be defined as the ability to select among two or more alternative events. The figure shows the executive atten- 
tional network proposed by Posner and coauthors (Fan et at., 2005). Notice that two lobes show most activation, 
the prefrontal and the parietal cortex. Attentional selection can change activity in many regions of the brain. Source : 
Holstege et al., 2004. 


et ah, 1999; Baars, 2002b). We will explore this question 
in later chapters on executive function and emotion. 

5*2 Executive and spontaneous attention 

You can decide to pay attention to a single word in 
this sentence, or the same word could be flashed on 
and off, so that it tends to compel your attention with¬ 
out executive control. Michael Posner has shown that 
the executive attention system in the brain involves 
prefrontal regions, as well as parietal ones, when spa¬ 
tial selection is concerned (Figure 2.22). 

The visual 'pop-out' effect is a common example of 
spontaneous attention (Figure 2.23). Some stimuli come 
to mind without voluntarily controlled visual search. 
When the same stimuli are embedded in a field of similar 
stimuli, the pop-out effect disappears and voluntary 
search becomes necessary. 



FIGURE 2.23 Spontaneous attentional capture. The task 
on both sides is to search for the horizontal green bar. On the left 
side, the green bar appears to 'pop out' spontaneously, with little 
effort. On the right hand side, finding the green target bar requires 
effortful search, which is thought to involve executive regions of the 
brain. Visual 'pop-out' also applies to salient stimuli - ones that are 
biologically or personally significant, like faces or human bodies, or 
those that are physically intense. Thus, we have two different kinds 
of selective attention: voluntary, goal-directed executive attention, 
and spontaneous, bottom-up attentional 'capture' by salient stimuli. 
Source : Squire et al., 2003. 
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FIGURE 2.24 In practice, conscious events are defined as events that people can report accurately. The 
functional diagram showing a hypothesized relationship between selective attention and conscious events. A 
number of scientists believe that selective attention may be needed for conscious sensory experiences. However, 
there are some contrary findings. Anew experimental literature has now grown to explore such questions. 


Attention is often thought to be required for con¬ 
scious experiences. We can represent this hypothesis 
in the functional diagram (Figure 2.24). As we will see 
in Chapter 8, this hypothesis is not always true, but in 
normal life conditions, attentional selection appears to 
lead to conscious experiences more often than not. The 
role of consciousness in human cognition has become 
a hot research topic, and we now have a good deal of 
evidence that helps to clarify the issue (see Chapter 8). 
Obviously, there is still a great deal that we do not 
know about consciousness, but it appears to be a pro¬ 
ductive topic for scientific research. 


6.0 ACTION 

The last elements of the functional diagram involve 
output: control of voluntary actions. There are some 
striking parallels between perception and action. Fuster 
(2003) points out that both input and output levels 
can be viewed as processing hierarchies. The visual 


hierarchy begins with retinal 'pixels' in cortical area VI, 
the primary visual cortex, and proceeds to areas spe¬ 
cialized for color, motion, and object recognition (see 
p. 167, Figure 6.11). On the output side, the motor hier¬ 
archy begins with general goals, influenced by emo¬ 
tional and motivational input from limbic regions. The 
most general goals are represented in more prefrontal 
areas and proceed down the motor hierarchy to sup¬ 
plementary and premotor regions which may trig¬ 
ger intentions and the urge to act (e.g. Penfield and 
Roberts, 1959). The primary cortical motor region (Ml) 
directly triggers movement of skeletal muscles. 

Figure 2.25 shows brain regions that become active 
in pushing a button with the right hand. The lower right 
panel shows a time scale marked in seconds, and antici¬ 
patory brain activity begins several seconds before the 
finger press. Notice that motor cortex is active on the left 
side, opposite to the hand that is commanded to move 
(this is called the contralateral side). However, motor 
cortex activates cerebellar activity on the same side 
as the moving hand ( ipsilateral ) until finally the finger 
press occurs at the zero point in the time scale. 
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FIGURE 2.25 What the brain does to push a button. Hulsmann 
et al. (2003) has shown how the voluntary goal of pressing a button rises 
over seconds before a spontaneous action. This snapshot shows the 
target time (time zero), when the finger press occurs (lower right). The 
two scans on the left show activation in motor regions of the left hemi¬ 
sphere, which controls the right hand. However, the upper right image 
shows activation on the right side of the cerebellum, which is required 
for fine motor movements like finger presses. This crossover activ¬ 
ity is consistent with the known anatomy of motor control. (Note that 
these brain images do not show other regions known to be involved in 
action control, including the basal ganglia and motor pathways.) Source : 
Hulsmann et al, 2003. 


FRONTIERS OF COGNITIVE NEUROSCIENCE 


The study of human memory in recent decades 


Cognitive neuroscience is an exciting new field encom¬ 
passing the formerly separate disciplines of biology and 
psychology, now united in the ongoing quest for under¬ 
standing the mind and the brain. An influential leader 
in this quest is Dr. Larry Squire, Distinguished Professor 
of Psychiatry, Neurosciences, and Psychology, at the 
University of California, San Diego School of Medicine, 
who has long studied the neural bases for long-term 
memory. 

We sat down with Dr. Squire and discussed the met¬ 
amorphoses of the study of human memory over the 
past several decades. Here are some highlights of that 
discussion. 

Humans have long been fascinated by our memories — 
how they are formed, how some memories from long 
ago still seem so vibrant to us today, and how they can 
be damaged or lost. The modern era of the study of 
memory began with a seminal paper published in 1957 
by Dr. Brenda Milner (Scoville & Milner, 1957). 

It all began when she and Dr. Wilder Penfield, the 
famed neurosurgeon, were studying two patients who 
had unilateral removal of structures in the medial tem¬ 
poral lobe as part of treatment for intractable epilepsy 
(Penfield & Milner, 1958). An unexpected result of the 
surgeries was that the patients experienced severe amne¬ 
sia: loss of memory. These case studies were presented 
at a scientific meeting in 1955 and shortly thereafter. 
Dr. Penfield received a call from Dr. William Scoville, a 
neurosurgeon from Hartford, Connecticut. It seems that 



FIGURE 2.26 Larry R. Squire, PhD, University of 
California and the Veterans Affairs Medical Center, San Diego, 
CA, USA. 
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he had a similar result — severe amnesia — in a patient 
who had medial temporal lobe structures removed 
bilaterally. Dr. Milner traveled to Hartford to study this 
patient, a young man of 29 who had suffered from epi¬ 
lepsy since childhood due to a bicycle accident. In 1953, 
his epilepsy was so severe that Dr. Scoville decided to 
surgically remove brain structures in the medial temporal 
lobe, including the hippocampus. The epilepsy was con¬ 
trolled, but the young man, known as patient HM, was 
never the same: his memory was forever damaged. HM 
became a cornerstone in the study of human memory 
and the role of the hippocampus. His death in December 
2008 allowed scientists to finally release his name and 
more details of his landmark case (Squire, 2009). 

Reports of the damage to HM's memory due to the 
surgery launched a veritable sea of new studies about the 
role of the medial temporal lobe and, specifically, the hip¬ 
pocampus, that continue to this day. Three fundamental 
principles emerged from the study of HM: (1) memory 
is a distinct brain function, separable from general cog¬ 
nitive and language functions, since HM's intellect and 
ability to produce and understand language were intact; 
(2) the medial temporal lobe is not necessary for immedi¬ 
ate memory, since this was largely normal in HM; and (3) 
medial temporal lobe structures are not the final storage 
location for memories, since although removal of these 
structures induced severe amnesia in HM that spanned 
many years in the past, he still retained memories of his 
childhood. Another key finding from the study of HM 
was that human memory was not a monolithic system: 
although clearly separable from general cognitive func¬ 
tion, memory nevertheless was comprised of subcatego¬ 
ries or systems. HM, although severely amnesic, was able 
to learn new tasks and procedures. Out of the careful 
study of what was impaired and what was intact in HM, 


Dr. Larry Squire and other memory researchers began to 
develop theories for separable aspects of memory. 

The next four decades produced a wealth of new knowl¬ 
edge about memory in the brain: early studies focused 
on lesion approaches, studying human patients who had 
damage to the medial temporal lobe and experimentally 
"lesioning" these regions in laboratory animals. The emer¬ 
gence of new techniques in molecular genetics and func¬ 
tional neuroimaging in the 1990s launched a new sea of 
investigations of the genetic and neural bases for memory. 

Dr. Squire has played a central role in these inves¬ 
tigations: using a multidisciplinary approach that 
encompasses lesion studies in human, lesion studies in 
nonhuman primates and mammals, genetic research, 
behavioral studies, and functional neuroimaging investi¬ 
gations, Dr. Squire has been a leader in the quest to elu¬ 
cidate the brain bases for memory. Recent findings are 
shedding new light on the functional role of the hippoc¬ 
ampus in memory. Cumulatively, these findings suggest 
that the hippocampus is initially highly involved with 
the cortex in the support of long-term memory storage 
but that this role gradually declines over the years. Thus, 
there is a graded change in the role of the hippocampus 
with respect to the final repository in the brain of long¬ 
term memories: this notion is well-supported by early 
observation that HM did recall early childhood memories 
despite his severe amnesia. Other new findings are related 
to sleep and the role of the hippocampus in memory con¬ 
solidation. Recent findings support an idea of a dialogue 
that takes place between the hippocampus and the cortex 
during sleep, which may reflect how recent memories 
become long-term and lasting (Squire, in press). 

Where do we stand today? Dr. Squire has developed 
an overarching theory for long-term memory systems 
and the brain structures that subserve them that has 
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FIGURE 2.27 Schematic /taxonomy of mammalian long-term memory systems. 
Source : Squire, 2004. 
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been highly influential in the study of human memory 
(Figure 2.27; Kandel & Squire, 2000; Squire, 2004). Where 
are we heading tomorrow? No one knows, but it is 
likely the development of new methods and techniques 
for investigating brain bases for memory will serve to 
jump-start our knowledge on the topic of memory. 
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7.0 CONSOLIDATION OF SHORT-TERM 
EVENTS INTO LONG-TERM MEMORY 

Clive Wearing was able to experience the present 
moment, but he could no longer store that experience 
in long-term autobiographical memory. Long-term 
stores are shown in the functional diagram along the 
bottom, ranging from perceptual memory to highly 
practiced habits (Figure 2.28). 


The question of where long-term memories are 
stored in the brain continues to be one of the myster¬ 
ies. Fiowever, there is a great deal of indirect evidence 
for the consolidation hypothesis , suggesting that memory 
is stored in the same areas that support active moment- 
to-moment brain functions. Working memory changes 
quickly and is vulnerable to interference because it 
depends on dynamic electrochemical activities in large 
populations of neurons and synapses. After enough 
rehearsal, however, the synaptic connections needed to 



FIGURE 2.28 Long-term stores are shown in the functional diagram along the bottom, ranging from per¬ 
ceptual memory to highly practiced habits. When memory stores are not activated, their contents are unconscious. 
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FIGURE 2.29 Transient memories become consolidated over 
time. Short-term memories are subject to interference, probably 
because they involve active neuronal circuits in the brain. If infor¬ 
mation is retained over a period of seconds to hours it may become 
permanent or 'consolidated'. A good night's sleep is now known to 
facilitate memory consolidation. Long-term memories are believed 
to require protein synthesis, which increases the efficiency of synap¬ 
tic connections. Source : Nader, 2003. 


Executive memory Perceptual memory 



RF 



(b) 

FIGURE 2.30 If the consolidation hypothesis is true, memory 
may be stored in many different regions of the brain by means of 
long-lasting synaptic connections. Fuster (2003) suggests, therefore, 
that the rear (posterior) half of cortex can be thought of as involving 
sensory memory systems, and the frontal half may involve execu¬ 
tive and motor memories. In addition, the hippocampal neighbor¬ 
hood is certainly involved with episodic memory (memory for 
conscious experiences), while subcortical areas like the basal ganglia 
and cerebellum play a role in motor learning. Fuster's hypothesis 
provides a useful way to think about the brain basis of long-term 
memory. Source: Fuster, 2003. (See p. 124 for details.) 


store memories are thought to become more efficient and 
longer lasting. Figure 2.29 suggests that dynamic synap¬ 
tic contacts are converted to more permanent connec¬ 
tions, using protein synthesis and RNA. The traditional 
slogan for the consolidation hypothesis is: 'Neurons that 
fire together, wire together' (see Chapter 3). 

A number of memory theorists propose that long-term 
memory traces may be stored in the same areas of the 
brain that are involved in their active forms. Figure 2.30 
by Joaquin Fuster illustrates this point about the cor¬ 
tex. The figure shows the posterior half of cortex 
(shades light blue) is mostly involved with sensory 
functions, and is thought also to store sensory and 
perceptual memories. As mentioned above, the fron¬ 
tal half involves executive functions, motor control. 


Broca's area, and other cognitive functions, and may 
therefore be involved in consolidating memory for 
those functions (see Chapter 5). 

The consolidation hypothesis provides a useful 
simplification of much evidence. Long-term memory 
is covered in detail in Chapter 9. 


7.1 Is working memory just re-activated 
permanent memory? 

The consolidation hypothesis raises an interesting 
question: is it possible that immediate memory just 
activates long-term traces encoded in the brain? We 
can look at this question from both long-term and 
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TABLE 2.3 Some brain hypotheses for the functional framework 


Sensory input and sensory stores 
Voluntary selective attention 

Spontaneous selective attention 
Verbal WM 

Visuospatial sketchpad 
Response output 
Transient storage of WM 
Long-term memory systems 


Sensory cortex (posterior half of cortex), as well as sensory pathways and thalamic relay nuclei 
Prefrontal and parietal regions may modulate sensory cortex to select some input signals over 
others 

Sensory regions may trigger orienting and selective signal processing 

Extended Broca and Wernicke's areas, prefrontal cortex, and medial temporal lobe 

Visual cortex, including the parietal lobe and prefrontal regions 

Prefrontal and motor cortex, basal ganglia, cerebellum and motor pathways 

Medial temporal lobe interacting with neocortex 

Lasting changes in cortical connectivity 


short-term memory perspectives. At this point, you 
may want to keep this possibility in mind - just to 
remind yourself that all of today's ideas in cognitive 
neuroscience may be seen from more than one point 
of view. It is important to stay open until the evidence 
is beyond dispute. 

Table 2.3 suggests some possible brain regions for 
the cognitive framework. Later chapters will explore 
these hypotheses in greater detail. 


8*0 SUMMARY 

This chapter has explored a broad functional frame¬ 
work for cognitive neuroscience, based on widely 
accepted ideas. We have also shown some brain cor¬ 
relates of the functional framework. 

Chapter 1 summarized some history about Broca's 
and Wernicke's studies of the speaking hemisphere, 
and this chapter shows how those regions seem to 
be involved in internal speech (verbal rehearsal). 
Similarly, visual imagery seems to make use of visual 
cortex, and even 'inner music' may involve brain areas 
involved in overt music listening and playing. As a 
simplifying hypothesis, we can therefore look for sim¬ 
ilarities between overt and covert cognitive functions. 
This tentative hypothesis will aid in understanding 
later chapters. 

Immediate memory seems to depend on the medial 
temporal lobe, including the two hippocampi and 
their surrounding regions. Damage to those regions 
impairs the ability to transfer information from the 
present moment to long-term storage. The consolida¬ 
tion hypothesis suggests that long-term memory may 
be a permanent strengthening of the active connec¬ 
tions established by current experiences held in work¬ 
ing memory. 

The rear half of cortex is involved in sensory proc¬ 
esses, and probably also in sensory-perceptual memory. 


The front half of cortex is involved with motor and 
executive functions, and probably also with long-term 
memory needed for those processes. Indeed, immedi¬ 
ate memory can be looked at the other way, in terms of 
long-term capacities that are evoked by current input. 

Selective attention has been studied for some fifty 
years and, in the last 15 years, the traditional question 
of conscious experience has again come to the fore. As 
we will see, it has been studied using many different 
experimental methods. Conscious cognition comple¬ 
ments the other features of the functional diagram (see 
Figure 2.24). 

It is important to hold these ideas lightly as a way 
of thinking about cognitive neuroscience. They will be 
tested in more detail in later chapters. 

There is ongoing debate about the meaning of terms 
like 'working memory', 'attention', and 'conscious 
experiences'. Such debates are common in science. We 
have a reasonable working consensus today, however, 
about empirical definitions - the kind of evidence that 
is believed to tell us about these concepts. In prac¬ 
tice, 'working memory' has come to mean any brief 
memory phenomenon, on the order of tens of seconds, 
that allows us to retain and manipulate information. 
Simply remembering a telephone number while reach¬ 
ing for the phone fits this definition. So does mental 
arithmetic, and thinking about the best route to take 
in walking from one place to another. Working mem¬ 
ory was first explored by giving people lists of unre¬ 
lated words or numbers, and asking them to recall 
them ten to thirty seconds later. Another popular 
approach is 'delayed match to sample', in which peo¬ 
ple are given a stimulus to remember and are asked to 
respond 'yes' or 'no' when they see it. 

Delayed match to sample (DMTS) is commonly 
used in animal studies, since we cannot ask rats or 
monkeys to report their experiences. Perhaps we can 
think of successful matching to a past event as a kind 
of reporting. Animal studies have been an important 
source of evidence. For example, they helped confirm 
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the role of the hippocampus in learning. In practice, 
the defining feature of 'working memory' is the ability 
to hold an item in memory for several seconds. This 
reasoning has been applied to working memory for 
odors and even planned eye movements. 

There has been a wealth of new knowledge learned 
about human cognition and the brain. It is an intricate 


business to tease apart these aspects of human thought 
and action experimentally. While the field has made 
progress in understanding how these areas of cogni¬ 
tion interact, we have far to go before we will have 
a clear understanding of the dynamic integrative 
and interactive processes that underlie the human 
condition. 


9.0 STUDY QUESTIONS AND 
DRAWING PRACTICE 

9.1 Study questions 

1 What have the cases of Clive Wearing and HM 
taught us about memory? 

2 What brain areas are believed to be involved in 
working memory and long-term memory? In 
visual imagery and spatial planning? 

3 What is inner speech and how does it relate to 
everyday cognition? 


4 Which brain landmarks could you use to tell 
where the eyes are looking? Where is the back 
of the head? The left side? The right side? 

5 What is a useful definition for working mem¬ 
ory? For selective attention? For the different 
types of long-term stores? 

9.2 Drawing exercises 

1 Label the Framework in Figure 2.31. 

2 Label the colored functional regions in the brain 
diagram in Figure 2.32. 



FIGURE 2.31 
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FIGURE 2.32 

















There is no more important quest in the whole of science probably than the attempt to 
understand those very particular events in evolution by which brains worked out that special 
trick that enabled them to add to the scheme of things: color , sound , pain , pleasure , and all the 
facets of mental experience . 


Roger Sperry (1976) 


Parietal 



How neurons are often organized. It is helpful to think of neurons 
in the cortex (and many other structures) as layered hierarchies. 
In the cortex, signals can flow in any direction. The figure shows 
a circle of such layered hierarchies, allowing communication 
between auditory, visual, and other regions. Source : Modified 
from Friston, 2003. 
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1.0 INTRODUCTION 

Chapter 2 gave a first overview of brain functions like 
perception, working memory, attention, and executive 
control. These have become easier to study since the dis¬ 
covery of brain imaging methods. But what do we know 
at the level of the neuron? Can we tell a plausible story 
at a more detailed biological level? 

We begin with some basic ideas about neurons and 
their connections. In Chapter 1, we pointed out that 
science always makes inferences that go beyond raw 
observations, using abstract concepts that 'make a 


believable story'. Neurons have been studied for two 
centuries, and the simplifications we will use here are 
well based in evidence. But, like any other simplified 
story, this one may encounter surprises. If it does, we 
must change it accordingly. 

Brains are made of signaling cells - neurons - which 
are highly conserved over evolution. That is, they have 
remained relatively stable over hundreds of millions of 
years, as suggested by the fact that very different animal 
species have similar kinds of neurons. In most respects, 
neurons are like other body cells, but they are highly 
specialized for electrochemical signaling: they accept 
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Motor 



FIGURE 3.1 Neurons, networks, and the brain. Three levels of description, from neurons to large "nets of nets'. Sensory 
and motor cortex are often viewed as processing hierarchies (Friston, 2003). In this example, visual maps in the occipital lobe 
go from simple (VI) to complex representations of the visual world, flowing deeply into the temporal and parietal lobes. The 
upper circle of hierarchies represents the overall architecture of the cortex. Source : Friston, 2003. 


input at their dendrites from other cells, and send an 
electrochemical signal along an output branch, the axon. 
The entire brain can be viewed as a hypercomplex sur¬ 
face of neurons and their connections (Figure 3.1). 

Dendrites and axons are thin micron-level tubes 
extruding from the cell body; an average neuron may 
have ten thousand input branches, and one or more 
output fibers (Figures 3.2 and 3.3). Nerve cells fire 
their spikes much more slowly than the electronic 
arrays that run computers, but they still do some 
things far better than current computers. Computers 
do not reach human performance at this time in terms 
of perception, language, semantic memory, action con¬ 
trol, or artistic creativity. 


1.1 Real and idealized neurons 

The brain is a kind of Amazon rain forest with many 
undiscovered species of trees, plants, and animals. To 
begin we will focus only on one prototypical tree, but 
this is only a convenient fiction. The great diversity of 
the neurons in the brain is suggested by Figure 3.4 - 
there are many classes of neurons, neurochemicals, 
and potential mechanisms of information processing. 
Our first simplification, therefore, is to focus only on 
an integrate and fire neuron (see Figure 3.3). This classical 
neuron accepts input from other nerve cells in its den¬ 
dritic branches with graded membrane potentials; i.e. 
the voltages across the membranes can have continuous 
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Axon 



FIGURE 3.2 A single neuron. A spectacular recent photomi¬ 
crograph of a single bipolar neuron. Cortical neurons may have ten 
thousand dendrites (input fibers) and one or more axons (output 
fibers). All fibers have smaller spines tipped with synapses con¬ 
necting to neighboring neurons. The photo does not show the three- 
dimensional bushy shape of the neuron. Source : Standring, 2005. 


values. The graded dendritic potentials add up and, 
if the total voltage over a brief time interval exceeds 
about —50 mV, they trigger a fast traveling spike 
or action potential in the axon of the nerve cell (see 
Figures 3.3 and 3.5). 

The classical neuron is thought to send its signal by 
firing spikes - sending action potentials from the cell 
body down the axon to the terminal buttons. At the 
terminals, a neurochemical messenger is released to 
diffuse across a very small synaptic gap. It then trig¬ 
gers a postsynaptic potential in the neighboring neu¬ 
ron. Figures 3.3 to 3.5 show an abstract version of this 
process. In cognitive neuroscience, we usually focus 
on the ways neurons connect and interact in networks. 
The prototypical neuron is a useful starting point. 



FIGURE 3.3 An idealized neuron. A simplified neuron with 
its dendrites on top, showing their spines as tiny knobs. Dendrites 
receive synaptic inputs that evoke graded membrane potentials 
(labeled Reception). When dendritic potentials rise above thresh¬ 
old in a very brief time interval (Integration), and are added to the 
intrinsic membrane potentials, they can trigger a fast depolarization 
of the axonal membrane - an all-or-none spike (Encoding). Spikes 
cause the release of neurochemicals at the axon terminals, which 
repeat the whole process, by evoking graded potentials in the next 
cell (Output). Source : Byrne and Roberts, 2004. 


Given the complexity of the brain, our ideas for 
understanding it are basically simple. But simple units 
combine to make up massive nervous systems. There 
are many ways to use nerve cells for a host of different 
receptors, pathways, circuits, networks, and effectors. 

1.2 Excitation and inhibition 

Classical neurons are connected by way of synapses 
(Figure 3.6), which can be excitatory or inhibitory. Thus, 
the probability that the next neuron will fire a spike can 
be either increased or decreased. A neuron in cortex 
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FIGURE 3.4 Two neurons connect across a synapse. Note that 
some synapses can feed back on the original neuron. Source : Byrne 
and Roberts, 2004. 


may have some ten thousand input synapses, and 
a dozen output synapses terminating on other neu¬ 
rons. Dozens of other factors determine the activities 
of neurons - the sleep-waking cycle, the availability 
of chemicals for making neurotransmitters, and more. 
These factors all affect the likelihood of a signal going 
between two neurons, and they can be summarized as 
synaptic weights , which represent the chances that one 
neuron will cause the next one to fire. Thus, the great 
variety of neurons is often simplified into an idealized 
neuron, the integrate-and-fire unit; and the many ways 
in which neurons can trigger each other is simplified 
into connection probabilities between neurons. 

Neurons have a great variety of shapes, branching 
patterns, and synapses. There are at least half a dozen 
major neurotransmitters , with at least thirty 'minor' ones, 
the neuropeptides. It is now known that electrical syn¬ 
apses, which use no neuro-transmitter at all, are much 
more common than was previously believed. Even the 
dendrites of a single nerve cell may be able to com¬ 
pute useful information. There is evidence that neuro¬ 
glia, the support cells for neurons, may also have an 
information processing function. Other surprises keep 
coming. For example, for more than a century it was 
believed that unlike other body cells, new neurons were 
not produced after birth. However, it is now known 
that stem cells - new, undifferentiated progenitor cells 
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FIGURE 3.5 Signals traveling along the axon, (a) Neuronal signals travel along axons through the exchange of positive 
and negative ions in adjacent regions of membrane. In the axon, region 2 is undergoing depolarization, while region 3 has 
already generated the action potential and is now hyperpolarized. The action potential or spike will propagate further by 
depolarizing region 1. (b) Myelinated axons are wrapped in specialized Schwann cells. The axon is only exposed at the nodes 
of Ranvier. (c) Action potentials in myelinated fibers are regenerated at the nodes. Myelinated fibers have higher conduction 
velocity than bare axons. Source : Ramachandran, 2002. 
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FIGURE 3.6 A basic synapse. Two cells in contact are labeled presynaptic (the lighter blue) and 
postsynaptic (darker blue). A spike in the presynaptic cell triggers release of a chemical neurotrans¬ 
mitter that diffuses across the synapse, and lowers the membrane potential of the postsynaptic cell. 
Some of the neurochemical machinery of this extraordinarily complex biological system is shown 
here. Source : Standring, 2005. 



FIGURE 3.7 An actual photomicrograph of a single neuron in the basal ganglia. Notice the dendritic spines - tiny 
protrusions on the dendrites, often containing end buttons for synapses. The horizontal bars mark 50 micrometers. Source : 
Paxinos, 2004. 
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for neurons - are born in some parts of the adult brain. 
Synapses are known to grow throughout the adult life¬ 
time, and small dendritic spines can grow in minutes to 
support new synapses (Figure 3.7). Thus, the standard 
neuron is only part of the whole picture. New evidence 
is bound to keep coming in for years to come. 

As Shepherd writes (2004): 

The idea of the nerve cell as an independent cellular 
unit ... continues to be one of the foundations of our 
concepts of nervous function. However, it has become 
clear that the whole neuron represents only one level 
in the organization of neural circuits and systems. 

We now recognize that nervous function arises out 
of many levels of organization, some of them within 
the neuron, some of them involving multineuronal 
interactions... 


1.3 Neural computation 

What justifies studying these simplified neurons? 
Perhaps more than anything else, it is the success of 
neural network models that have used simplified 
neurons in the last few decades. Artificial neural nets 
(ANNs) have been used to model many of the func¬ 
tions the brain performs - to recognize patterns, to 
plan actions in robots, learn new information, and use 
feedback to improve performance. Most ANN simu¬ 
lations are relatively small-scale and limited. None of 
them come close to the massive complexity of a liv¬ 
ing brain. But, for some jobs, ANNs have been more 
successful in doing human-like tasks than computer 
programs that use logic and standard mathematics. 
It is important to remember that ANNs are artificial. 
They are not the real thing, but they give us a greater 
understanding of the ways neural computation 
might work. 

In the history of science, new mathematical tech¬ 
niques often help to understand new questions. Neural 
computation similarly helps to understand nervous sys¬ 
tems. A great deal of progress has occurred on the basic 
mathematics of neural networks. There seems to be a 
small number of basic architectures - i.e. arrangements 
of simple units - with similarities to the networks found 
in the brain. 

Cognitive neuroscientists commonly focus on 
biologically plausible neural net models, those that are 
based on the known properties of a specific set of 
neurons and their connections. However, artificial 
neural nets often provide useful approximations to the 
reality. 


In addition to the simplified neuron of Figure 3.3, 
we will also use simplified synapses. As mentioned 
above, we will assume there are only two kinds of 
synaptic connections, excitatory ones (which increase 
the chance of triggering the next neuron), and inhibi¬ 
tory ones (which decrease that chance). Glutamate, the 
most common neurotransmitter in the brain, is known 
to be excitatory. GABA (gamma-amino butyric acid) is 
the most common inhibitory neurotransmitter. So our 
simplification is a part of neural reality. 

2.0 WORKING ASSUMPTIONS 

We have discussed using simple and idealized neu¬ 
rons in order to describe the basics of neural firing 
and connectivity. We will use these basic ideas already 
laid out as working assumptions as we continue our 
discussion of neurons and their connections. To sum¬ 
marize our working assumptions, they are: 

1 The integrate-and-fire neuron. We will assume that 
neurons work by adding graded voltage inputs 
until the total membrane voltage on the target neu¬ 
ron goes past a threshold value (approximately 
—50mV in real neurons). If it does, an all-or-none 
spike fires down the output branch, the axon. 

2 Connections are either excitatory or inhibitory. We 
can therefore assign a single number to the prob¬ 
ability that neuron A will trigger neuron B. That 
number is called the weight of the connection. 
As probabilities, connection weights have numbers 
between -1 and +1. Plus 1 means a 100 per cent 
certainty of triggering the next neuronal spike, 
and -1 is 100 per cent certainty of stopping 
the next neuron from firing. Weighting schemes 
can be more complex, but this is a useful first 
approximation. The vastly complicated neuro¬ 
chemistry can be reduced to just one number for 
the probability of transmission, the 'weight' of 
each connection. 

3 A second reason for the simplified neuron and 
synapse is that the basic mathematics of networks 
appears to be very successful in simulating cogni¬ 
tive functions. Neural nets can simulate the kind of 
pattern recognition that sensory systems perform, 
although our models are much simpler than any 
brain mechanism known today. Neural nets can 
learn and store information in much the way real 
neurons are believed to. Since the basic types of 
ANNs were worked out in detail over the last several 
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decades, they have been applied to many practical 
uses, from computer-based face recognition to pre¬ 
dicting the stock market. That is also scientifically 
important, because it shows that we understand 
enough about networks to begin to see how the 
brain might do those things. As always, we must be 
prepared to see the limits of our knowledge; but we 
have a basis for departure. 

4 Neurons can form one-way pathways, such as 
the optic nerve to the visual thalamus (the lateral 
geniculate nucleus). However, one-way pathways 
are quite rare. More likely, neurons run in two 
directions, forming two-directional pathways and 
networks, in which activity at point A triggers 
activity at point B, and vice versa. This is often 
called re-entrant connectivity (Edelman, 1989). 

5 As we will see, the nervous system loves arrays 
of neurons, often called maps. The cerebral cortex 
is a massive six-layer array, with an estimated ten 
billion cells and trillions of synaptic connections 
between them. The retina at the back of the eye 
is another array, this time with three layers (see 
Chapter 6). In fact, all sensory surfaces are arrays 
of receptors and their closely linked layers of relay 
and processing cells. Other arrays are found in the 
sensory thalamic nuclei, in the superior colliculi 
(which control eye movements), and in arrays of 
cells that control muscles. The brain's liking for 
arrays and maps, with dense connections between 
them, is another useful working assumption. 

6 Hebbian cell assemblies : when neurons combine by 
triggering other neurons, the resulting pattern of 
activity may be stable or unstable. Unstable pat¬ 
terns tend to die out, while stable patterns remain 
for some period of time. Such stable patterns are 
often called cell assemblies , and the term 'cell assem¬ 
bly' is often attributed to Donald O. Hebb (Hebb, 
1949). Hebbian cell assemblies may involve neigh¬ 
boring cells, or they may involve cells that are far 
away from each other. Cell assemblies that com¬ 
bine both excitatory and inhibitory connections 
tend to be more stable and lasting, as we will see. 
In the brain, transient connections are thought to 
be mainly electrochemical, while more lasting ones 
are thought to require protein synthesis. 


2 A Starting simple: receptors, pathways, and 
circuits 

While reflex circuits can be triggered by outside 
stimuli, they are normally integrated seamlessly into 


voluntary, goal-directed activities. For example, you 
can turn your head from side to side while reading 
this sentence. That is, you can follow a voluntary goal 
(stated in the previous sentence), and your oculomo¬ 
tor system will still keep your eyes focused on the 
moving window of the words you are reading at this 
instant in time. It is a remarkable achievement of sen¬ 
sorimotor adaptation, and most of the time it is quite 
unconscious and automatic. Oculomotor coordination 
is not just a simple reflex arc. 

Voluntary brain mechanisms, guided by explicit 
goals, are associated with cortex in humans. Very 
sophisticated subcortical circuitry is also engaged in 
planning and executing actions. Spinal centers may 
carry out commands from higher centers using sen¬ 
sorimotor reflexes, but they also return feedback 
signals to the brain. All these levels of control have 
endogenous (internal) as well as exogenous (sensory) 
input, both conscious and unconscious (Goodale and 
Milner, 1992). Thus, while there are certainly some 
simple reflex circuits like the famous knee-jerk reflex 
in Figure 3.8, reflexes rarely work in isolation. They 
normally work in the service of cortical goals. 



FIGURE 3.8 A simple reflex circuit. If you drape one leg over 
the other and tap just below the kneecap, your lower leg will jump 
out. This is the famous knee-jerk reflex, technically called the patel¬ 
lar tendon reflex. It is a classical example of a spinal reflex, control¬ 
led by a simple circuit. Sensory neurons pick up the mechanical tap, 
and transform it into a neural signal which is sent to the spinal cord. 
There, an interneuron links the sensory impulses to motor neurons, 
which contract the muscles in the upper thigh, making the lower leg 
jump outward. Source: Standring, 2005. 
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FIGURE 3.9 Receptors transform external energy patterns into 
neuronal activity. Although these receptors belong to different sen¬ 
sory systems, they are similar in structure and function. All convert 
physical energy from the environment into neural signals. Source : 
Hendry et al. in Squire et al, 2003. 


Reflexes are innate circuits. They emerge during nor¬ 
mal development and some disappear after some time, 
like a baby's innate grasping reflex, which allows it to 
take strong hold of an adult's finger. But learned non¬ 
voluntary processes are not reflexes. They are complex, 
interactive routines called automatic processes - highly 
practiced skills and habits. For example, as a skilled 
reader you automatically process the difference between 
the letters 'b' and 'p', even though they may look simi¬ 
lar to non-readers. In speaking and typing, your mus¬ 
cle control is very fast, highly organized, and mostly 
unconscious. Humans use numerous automatic proc¬ 
esses to carry out normal actions. Such learned autom¬ 
atisms are typically unconscious, effortless, and not 
under detailed voluntary guidance. They should not be 
confused with reflexes. Automatic skills start out under 
cortical control, but after sufficient practice they tend to 
come under subcortical control (Haier et al., 1992; Chein 
and Schneider, 2005). 

Each sensory nerve may actually contain paral¬ 
lel channels, conveying somewhat different features 
of the world. Thus, vision has a color pathway called 
parvo-cellular (because it has somewhat smaller cells), 
and a separate bundle of 'black and white' neurons 
called magnocellular (meaning small cells) (Figure 3.9). 


Similarly, the somatosensory pathway combines paral¬ 
lel channels for light touch, for heat, pain, and so on. 

Most sensory nerves stop off at the thalamus, where 
they contact synaptic relays on their way to cortex 
(Figures 3.10 and 3.11). These thalamic nuclei pro¬ 
vide point-to-point connections. Vision and touch are 
topographically organized, projecting receptor maps 
into higher level maps, well into the cortex. Audition 
tends to be more tonotopically organized, with neuron 
arrays corresponding to sound frequencies. 

Most sensory and motor pathways split and cross 
over the midline of the body on their way from the 
periphery to cortex. The evolutionary reason for cross¬ 
over is still mysterious, but it is a pervasive feature in 
humans and other mammals. 

While we tend to think of input-output pathways 
as carrying one-way signal traffic, this is actually rare. 
Most pathways run in both directions. For example, 
while there is a one-way flow of signals from the ret¬ 
ina to the visual thalamus, the next stage goes in both 
directions. Thus going from thalamus to VI, about 
90 per cent of the neurons are running the wrong way 
(i.e. from VI downward to the visual thalamus)! In the 
auditory system, the downward flow of neural signals 
goes to the very receptors themselves (see Chapter 5). 

This two-way signal traffic means that much of 
the central nervous system should be viewed not as 
simple traffic arteries, but as re-entrant loops - equiv¬ 
alent to neural networks with two or more layers 
(Figure 3.11). Edelman (1989) and colleagues have 
particularly emphasized re-entrant processing as a 
basic feature of the brain. From this point of view the 
brain is a vast collection of mutually echoing maps 
and arrays. 

2.1.1 Receptive fields and lateral interactions 

Lateral inhibition was first proposed as a hypothesis by 
Ernst Mach, a German physicist in the 1860s, based 
on the tendency of continuous visual gradients to be 
perceived as discontinuous. It was a brilliant hypoth¬ 
esis that has since been confirmed by directed test¬ 
ing (Figure 3.12). Lateral inhibition seems to be used 
in many places in the brain (Figure 3.13). In the retina, 
neighboring cells can inhibit each other, so that a tiny 
point of light on one cell will tend to stand out in con¬ 
trast to the adjacent ones (Figure 3.14). In touch, neigh¬ 
boring cells in the skin also use lateral inhibition. At 
higher levels in the brain, similar semantic concepts 
may have the effect of inhibiting each other, so that 
'astronomy' and 'astrology' might not be confused. Like 
the other neural strategies discussed in this chapter, the 









3.0 ARRAYS AND MAPS 




Pons 


Medulla 


Spinal 

cord 


nucleus 

* 


Nuclei of 
the lateral 
lemniscus 


Cerebral 

cortex 


Midbrain 


Ventropostero- 
lateral nucleus 


Coclea 


Superior 

olivary 

nuclei 


Somatosensory 

cortex 


Medial 

geniculate 

nucleus 


Interior 

colliculus 


71 


Visual 



(a) Somatosensory 


(b) Auditory 


(c) Visual 


FIGURE 3.10 Similarities between sensory pathways: body senses, hearing, and vision. All the senses begin with arrays of recep¬ 
tors, like the layers of touch receptors of the skin and the array of light receptors of the eye. After local processing, sensory information 
is condensed into a single nerve pathway, a compact bundle of neurons, which carries sensory signals to cortex. Note that these three 
sensory pathways stop off in thalamic relay nuclei, marked in green. All three pathways also split and cross over to the opposite side on 
their way to the cortex, called decussation. Source : Hendry et al. in Squire et al ., 2003, Elsevier. 


brain often borrows successful tricks and tips from an 
earlier evolutionary form and may translate it into later 
adaptations. 


3.0 ARRAYS AND MAPS 

As we have pointed out, neuronal arrays are widely 
found in the brain. An array is a two dimensional grid 
of neurons. When arrays represent a spatial pattern, 
they are often called maps. Spatial maps are a form of 


spatial coding in neurons. The brain performs tempo¬ 
ral as well as spatial coding, and may have a number 
of other ways of representing and processing informa¬ 
tion. Nevertheless, spatial maps are the most obvious 
kind of neural code (Figures 3.15 and 3.16). 

The retina itself can be thought of as a three-layered 
map of light receptors and their neighboring neurons, 
including ganglion cells, which send their axons from 
the retina to the visual relay cells of the thalamus 
(the lateral geniculate nucleus) (see Chapter 6). The 
thalamic visual cells then project their fibers to the first 
visual area of the cortex, area VI of the occipital lobe. 
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(b) Auditory (c) Somatosensory 


FIGURE 3.11 Sensory regions interact with thalamic nuclei. The thalamus is often 
called the 'hub' of the brain. It is not the only hub, but perhaps the most important. 
Notice that vision, hearing, and touch pathways all stop off in the thalamus on their 
way to their cortical projection regions. However, information is constantly bounced 
back from the cortex to various thalamic nuclei, so that there is a constant flow of sig¬ 
naling between all parts of cortex and thalamic nuclei. In some cases, the thalamus 
amplifies cortical activity, while in others it blocks or inhibits it. Note the striking simi¬ 
larities between cortical input and output layers in these three senses. Source : Alitto and 
Ursey, 2003. 



FIGURE 3.12 Lateral interactions. The brain often uses the same strategy in many different places. On the left is a "center 
surround' cell of the visual thalamus (the lateral geniculate nucleus). Light falls on the red circle in the middle but no light 
falls in the surrounding ring. Neurons pick up both light input and the absence of surrounding light, and enhance the contrast 
by mutual inhibition in the same layer, called lateral inhibition. The same mechanism is used in the center image, and even in 
the barrel cortex of the rat, the place in cortex where its whiskers project their neurons in a very simple, one-to-one fashion. 
Barrel cortex is often used to study cortical processing for this reason. Adjacent whiskers also show lateral inhibition, which is 
also found in touch, hearing, and even attentional control in the human brain. Source : Alitto and Ursey, 2003. 


VI also looks like a detailed map of the visual input, 
with different quadrants of the visual field projected 
to different parts of this region. From area VI, most of 
the visual information projects to other 'maps' in the 


cortex, some of them in the visual cortex itself, but also 
beyond it. While the resolution of spatial details tends 
to blur after VI, higher-level visual maps respond to 
more complex and abstract stimuli, like faces, objects. 
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FIGURE 3.13 Visual demonstrations of lateral inhibition. Lateral inhibition was first proposed 
as a hypothesis by Ernst Mach, a German physicist in the 1860s, based on the tendency of continuous 
visual gradients to be perceived as discontinuous. It was a brilliant hypothesis that has since been con¬ 
firmed by directed testing. Notice that lateral inhibition also applies to adjacent black squares, to color 
perception between opponent colors, and even to the perception of 'train tracks stimuli' which signal 
spatial depth to the brain. Source : Eagleman, 2001. 
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FIGURE 3.14 Center-surround receptive fields in sensory cortex. Mapping of somatosensory (tac¬ 
tile), visual, and hearing space is accomplished in part by neurons with center-surround receptive fields 
that are sensitive to specific aspects (location, color, frequency) of a sensory stimulus. 
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FIGURE 3.15 The visual input projects to cortical areas. In a 
classic experiment, Tootell et al. (1996) showed that a circle around 
the center of visual fixation evokes corresponding activity in the 
first visual projection area. Area VI. The authors made use of the 
fact that VI expresses a topographical visual input, and that circles 
are symmetrical around the fixation point. We are looking at a flat¬ 
tened projection of area VI. Source : Tootell et al ., 1996. 


visual scenes, attentional, emotional, and perhaps 
even decision-making networks, as they work their 
way forward in the cortex (Figure 3.17). Thus vision 
involves what looks like a large and widespread set 
of maps of the light input to the eyes. While this view 
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FIGURE 3.16 Visual quadrants map to cortical quadrants. 
Notice that the first visual projection area VI represents different 
visual quadrants in different regions of cortex. The bottom right 
quadrant projects into VI in the left hemisphere, in the upper half of 
VI. The early visual areas are folded tightly into the occipital cortex, 
around a fold called the calcarine sulcus. Source : Zeki, 2003. 
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FIGURE 3.17 Visual maps in the macaque. The rhesus macaque monkey's visual brain is often studied because of its 
similarities to the human visual brain. On the left are the major visual pathways in the macaque cortex. Each one flows from 
topographical map to topographical map. Starting with area VI, the information flows to later 'V' areas while preserving 
the topography of the input. The upper pathway is sensitive to location, and is therefore called the 'Where' pathway. The 
lower pathway is sensitive to color, shape, contrast, and object identity, and is called the 'What' pathway It is believed that 
activity in the 'What' pathway is directly involved in conscious visual experiences, perhaps beginning in the object recogni¬ 
tion area (IT). The 'Where' pathway may provide spatial frameworks for the 'What' pathway. It may not be directly con¬ 
scious, but it shapes the perceived location of objects. (RF = receptive field.) Source: Lamme and Roelfsma, 2000. 












































3.0 ARRAYS AND MAPS 


75 


is undoubtedly too simple, it captures a good deal of 
evidence. 

The body senses, like touch and pain perception, 
also project to map-like regions of cortex. Other senses 
like hearing, smell, and taste are less spatial in nature, 
but the auditory cortex has map-like regions organ¬ 
ized by pitch, like the strings of a harp. Thus, even 
the non-spatial senses show regular neuronal arrays 
and map-like regions. Finally, information from spe¬ 
cific sensory systems is combined in the parietal 
cortex, using spatial arrays that combine auditory, 
visual, touch information into map-like regions. These 
include a body-centered map (called egocentric) and 
an object-centered spatial array (called allocentric). 
It seems as if our brains like to organize a vast amount 
of incoming information in arrays that mirror the lay¬ 
out of the spatial surroundings. And motor cortex, as 
you might guess, looks much like a distorted map of 
the output systems, the skeletal muscles of the body 
(see Chapter 5). 

It is tempting to think that with the sensory half of 
cortex (the posterior half) using so many maps, there 
must be a little person inside looking at the maps. But 
this is the 'homunculus fallacy', as philosophers call it. 
The trouble with this idea is that it explains nothing, 
but merely moves the question to another level: Does 
the homunculus have its own brain, with its own spa¬ 
tial maps, and another, even tinier homunculus sitting 
inside? 

The question for scientists is therefore how to 
make sense of the great number of spatial and other 
neuronal arrays in the brain without resorting to the 
fallacy of supposing that we all have a little creature 
inside, looking at all the neuronal maps. Neural net¬ 
work models provide one set of answers today, as we 
will see. 

3*1 Maps flow into other maps 

The nervous system often uses layers of neurons in 
giant arrays; indeed, the entire cortex is a massive 
six-layered array containing about ten billion nerve 
cells. Neuronal hierarchies are stacked arrays of nerve 
cells. 

3*2 Neuronal arrays usually have two-way 
connections 

There is one very important surprise in this tidy pic¬ 
ture of maps flowing into higher-level maps of the 
visual input: after the thalamic nucleus, most axons 



FIGURE 3.18 Two-way traffic between arrays. Two-way traf¬ 
fic is the norm in the brain. For that reason, it makes more sense to 
think of the arrays of visual regions as layers of a two-way network, 
rather than one-way paths from point to point. Notice that lower 
maps are sensitive to simpler stimuli, while higher ones show faces 
and cars. However, there is constant predictive flow of information 
from higher to lower maps, to make it easier to identify all levels of 
description. Source : Ahissar and Hochstein, 2004. 


run downward from cortex to the thalamus. Some 90 
per cent of the axons between LGN (lateral geniculate 
nucleus) and VI send spikes downward to LGN, rather 
than upward to V3. Higher up, there is always two- 
way traffic of signals between VI and V2, V2 and V3, 
and onward. This might seem pretty odd for a simple 
transmission idea of the visual pathway. But we will 
see that bi-directional flow of spikes is the norm in the 
brain, not the exception (Figure 3.18). There are very 
few one-way streets in cortex. This is a very important 
point, which compels us to think of neuronal signals in 
a richer way. We will explore two-way transmission in 
more detail below. Receptive fields of sensory neurons 
become larger and more complex as we go up in the 
visual hierarchy. 

3*3 Sensory and motor systems work together 

Finally, while there are very clear anatomical divi¬ 
sions between sensory and motor pathways, they are 
constantly interacting. When we speak, we also hear 
ourselves speak. When we walk, an array of visual 
information streams across our retina. Video games 
that simulate the flow of optical vistas really given 
a sense of motion, even though they are only visual. 
The brain is constantly updating its motor systems 
by means of sensory input, and telling the sensory 
systems what to expect by way of motor signals. 
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FIGURE 3.19 Sensory and motor hierarchies are themselves 
arranged in hierarchies. Fuster (2004) suggests that all of cortex 
can be seen in terms of cycling sensory and motor hierarchies, with 
information flowing between the posterior (sensory) regions and 
the frontal (motor and planning) areas. Early sensory cortex flows 
into unimodal association areas, which interact with premotor cor¬ 
tex, which is believed to encode the 'urge' to perform an action. 
Higher up the sensory hierarchy, polymodal association areas com¬ 
bine hearing, touch, and vision, and interact with the forward part 
of the pre-frontal cortex. Notice that there is constant exchange of 
information through the environment as well. Thus, we can hear 
ourselves speaking, and we can see a flow of visual vistas when we 
walk. Source: Fuster, 2004. 



FIGURE 3.20 A step pyramid as a hierarchy. One useful image 
for a brain hierarchy is a step pyramid, like the Inca city of Machu 
Picchu. While the levels are stacked on top of each other, signaling can 
take place through many different pathways in the hierarchy. Thus, the 
people exploring the step pyramid can walk up, down, or sideways. 


Fuster's classical diagram of the cortex makes this 
point emphatically (Figure 3.19). Functionally, the 
nervous system is always cycling information between 
input and output channels, to keep the sensory and 
motor world in synchrony (Fuster, 2004; Kandel et al., 
2004). 

Fuster (2004) suggests that both front and back can 
be viewed as massive hierarchies of local hierarchies, 
starting from sensory receptors, and becoming more 
and more general as information flows upward in the 
sensory hierarchy (see Figure 3.19). The motor hierar¬ 
chy can be viewed as going in the opposite direction, 
ending up at motor neurons. However, as the diagram 
indicates, more and more information is exchanged 
between the two hierarchies in an ongoing perception- 
action cycle, from a low level (as in listening to one's 
own voice speaking) to a very high level of planning, 
thinking, and anticipating the future (see Figure 3.20 
for an image of an architectural hierarchy). 


Friston (2003) has published a useful diagram for 
a processing hierarchy (Figure 3.21), which we will 
adopt here. Each neuronal array is called a 'map' in 
this version, and while maps exist at different levels, 
signals may travel up, down, and laterally. We will see 
some examples in later chapters. 

Notice that neural net models sometimes use dif¬ 
ferent directional names to refer to the same idea. 
'Bottom up' is the same as 'feedforward'. 'Top-down' 
flow is often called 'feedback'. We can simply flip the 
hierarchy on its right side to make 'bottom up' look 
like 'feedforward'. 


3*4 Temporal codes: spiking patterns 
and brain rhythms 

Arrays, maps, and hierarchies support spatial coding 
in neurons. But the brain makes use of temporal coding 
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FIGURE 3.21 An abstract hierarchy. Notice that like the step pyramid above, this hierarchy 
allows information to flow in all directions. This is a typical layout for sensory and motor hierarchies. 
Source : Friston, 2003, redrawn. 
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FIGURE 3.22 Neurons have different spiking codes. While 
it is easy to visualize the map-like spatial coding in the brain, neu¬ 
rons also code information over time. The two electrical traces show 
the voltages of simulated thalamic neurons. These neurons have 
two different spiking codes (McCormack and Huguenard, 1992). 
Source : White, 2002 in Ramachandran, 2002. 


as well. Figure 3.22 shows an example of a single 
neuron in the thalamus which shows two different 
spiking patterns. The timing of a neuronal spike rela¬ 
tive to other neurons can also convey information, 
much as a clock shows the time of the present moment 
relative to the 24-hour cycle. Neurons like this may 
serve as pacemakers for large populations of cortical 
neurons during delta sleep, for example (Figure 3.23). 
Another hypothesis suggests that fast firing of tha¬ 
lamic neurons may trigger waking when the sound 
of a loud noise or a crying baby is detected, or some 
other significant stimulus. Arboreal primates, for 
example, need to respond very quickly if a tree branch 
breaks. 

Individual neurons also exhibit spiking codes. For 
example, the auditory nerve performs frequency cod¬ 
ing (see Chapter 5). Every time the eardrum vibrates. 
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FIGURE 3.23 How single neurons are recorded in living ani¬ 
mals. The needle electrode causes no pain, since the brain itself has no 
pain receptors. It is implanted using a scalp attachment that allows the 
cat to move comfortably. The electrode picks up 'spikes' from a sin¬ 
gle neuron, which are amplified and shown on the upper right. Trials 
over time are shown on the colored screen. Source : Tsodyks et at., 1999. 


three tiny bones in the inner ear transfer mechanical 
ripples to a fluid, which in turn moves hair cells on the 
basilar membrane. The hair cells are the auditory recep¬ 
tors, which fire whenever they are mechanically stimu¬ 
lated. Together their axons make up the auditory nerve 
and, so long as the incoming sound frequency is fairly 
low, the firing rate of the auditory nerve follows the 
movements of the eardrum. The auditory nerve there¬ 
fore shows a simple one-to-one frequency code. It is a 
temporal code rather than a spatial map of input (see 
Section 3.5). 

When we look at the electrical activity of tens of bil¬ 
lions of neurons, the entire brain seems more like an 
orchestra than like a single piccolo. After hundreds of 
millions of years of evolution, it seems likely that the 
brain has evolved neurons with many kinds of tempo¬ 
ral and spatial codes. 


BOX 3.1 Some ways to visualize neural communication 


Neuronal communication can be visualized using stand¬ 
ard graphics. 

Figure 3.24(a) shows a cross-correlogram reflecting 
the activity of one neuron depending on a related one. 
Sherman and Guillery (1985) recorded spikes from 
a target neuron (postsynaptic) when a spike occurred 
in an input neuron (presynaptic). We are therefore 
looking at the efficiency of spike signaling across a syn¬ 
apse. Sherman and Guillery recorded in a part of the 
brain where a one-to-one relationship can be found 
between two neighboring neurons. The presynaptic cell 
is located in the visual thalamus, which relates point- 
to-point to cells in the primary visual cortex, area VI 
(see Chapter 6). 

The vertical axis indicates the number of spikes in the 
postsynaptic neuron, given a previous spike in the pres¬ 
ynaptic neuron at time t = 0. Most neurons show a con¬ 
stant background rate of firing, so that we are looking for 
activity that is significantly above the background rate. 
In this example the activity of the two neurons is very 
closely linked. Within a few milliseconds after the presy¬ 
naptic neuron fires, the corresponding postsynaptic cell 
also fires. We are therefore seeing high-fidelity transmis¬ 
sion of a spike signal across a synapse. 

Figure 3.24(a) is the simplest example of how two neu¬ 
rons can covary. They often interact in much more com¬ 
plex ways, of course. Cross-correlograms can be calculated 
between any pair of neurons that can be recorded. 

Figure 3.24(b) shows that neural communication can be 
observed on the scale of the entire brain. In this case we 
are looking downward at the cortex displayed as a circu¬ 
lar disk, with the eyes and the nose of the subject on top 


of the circle. The black lines represent gamma synchrony 
between the connected points, as recorded by intracranial 
electrodes. (Intracranial recording often is done in con¬ 
scious epileptic patients before surgery.) Gamma rhythms 
are defined as brain rhythms in the EEC band between 25 
and 120 Hz. 

Doesburg et al. (2007) presented visual flashes to the 
left and right of the visual fixation point in conscious epi¬ 
leptic patients. Figure 3.24(b) shows the case of a visual 
cue presented on the left side of the visual field. Because 
of the cross-wiring of the visual pathway, a left visual 
cue triggers activity in the right occipital cortex, where 
visual signals are detected and identified. The occipital 
cortex thereupon generates gamma activity that seems to 
recruit resonant "echoing" in many other parts of the cor¬ 
tex, as shown by high gamma synchrony between distant 
regions. Such task-related synchrony now has been found 
in many cognitive tasks (see Chapter 8). 

Finally, Figure 3.24(c) shows time-frequency graphs of 
brain activity after a visual face is presented, obtained by 
Lachaux et al. (2007), again using MEG and intracranial 
recording in the lateral occipital gyrus, a part of the 
visual cortex. In these graphs the vertical axis represents 
brainwave frequency, and the horizontal axis shows 
the time in milliseconds after some event of interest, like 
the presentation of a face. The colors in the color bar 
represent the intensity of electrical activity (called 
power), with white and yellow representing the highest 
levels. 

As you can see, there is a marked burst of high power 
beginning almost immediately after time t = 0. The fre¬ 
quency range of the yellow and white burst is 50-150 Hz 
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on the vertical axis, and about 100-500 milliseconds after 
the onset of the visual stimulus. Time-frequency graphs 
are used widely in cognitive neuroscience, and it is 
important to understand them. 

The black marks at the bottom of Figure 3.24(c) indi¬ 
cate a decrease of power in the alpha band of 8-12 Hz. 
Decreased alpha often is seen in visual tasks. 


Neuron-to-neuron cross-correlogram: 
across a single synapse. 



(a) Time after visual stimulus (ms) 


FIGURE 3.24(a) A cross-correlogram relating the firing 
of a thalamic neuron to the corresponding neuron in the visual 
cortex. Source : Adapted from Sherman and Guillery, 1998, with 
permission. 
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FIGURE 3.24(c) A time-frequency graph used to display 
the results of neuronal signaling studies in the brain. It allows 
us to see a frequency range of electromagnetic activity over a 
precise time period after a stimulus is presented, with colors 
indicating the intensity of the activity. This graph shows a high 
power burst (white and yellow colors) in the gamma range 
(about 50-150 Hz in this case), beginning about 100 ms after 
the onset of the face presentation, and dropping rapidly after 
—500 ms. The dark traces along the bottom of the graph indicate 
a loss of signal power in the theta and alpha range of 4-12 Hz for 
about a second. The horizontal black bar below the graph shows 
the duration of the visual stimulus, a "mooney face" (Chapter 8). 
Source : Lachaux, J. P., et al., 2007. 




Start : Left visual cue 


Finish: 

Widespread burst of gamma synchrony 
with other cortical regions, from right 
occipital cortex at time t = 300 ms 


Right Hemisphere: visual 
stimulus is detected 


FIGURE 3.24(b) A top-down head view of 
intracranial EEG in response to a visual cue in the left 
visual field. The left image shows the path taken by 
neuronal signals corresponding to the visual cue in 
the left visual field. Notice the dark lines on the image 
of the head (right) showing high gamma synchrony 
between visual cortex and other cortical regions, both 
in the same and opposite halves of the cortex. Gamma 
synchrony seems to play a role in cross-regional sig¬ 
naling in many cognitive tasks. Source : Adapted from 
Doesburg et al, 2007, with permission. 
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3.5 Choice-points in the flow of information 

If the brain had only straight highways it would be 
easy to understand. Instead, it has pathways with 
many choice-points, in which traffic can flow right or 
left, or jump a whole level forward or backward. That 
is the point of the step pyramid in Figure 3.20: it looks 
like a staircase, but hikers can go up or down, laterally, 
or in more complex interactive dance patterns. 

How do we know there are choice points in neuro¬ 
nal traffic flow? The anatomical connections indicate 
as much, but the fact that humans and animals con¬ 
stantly deal with ambiguities is an important source 
of evidence as well. Most words in natural language 
have more than one meaning (a fact we rarely notice 
consciously, but a glance at a good dictionary will 
show it). Dictionaries understate the degree of ambi¬ 
guity in language because they are used by skilled 
native speakers who bring a great deal of contextual 
knowledge to their reading of verbal definitions. 
There is much that a dictionary writer can simply take 
for granted - but the nervous system cannot. 

In the visual world we also encounter constant 
ambiguities that we do not bother to make conscious. 
Daylight changes color at sunset. When we walk 
through woods the light is filtered through green 
leaves. Wearing sunglasses filters colors, and even 
glass windows create reflectance patterns that change 
the visual world. Those differences rarely become 
conscious, because the visual system tends to main¬ 
tain color constancy, sometimes from memory. But in 
the brain, those differences must be processed, and 
separated from color signals that we need to notice. 
Animals cannot afford to miss a predator stalk¬ 
ing through the tall grass just because it's sunset, or 
because there are shadows from surrounding trees, or 
because there is an early morning mist. We need excel¬ 
lent visual brains to survive. 

The famous Necker cube is shown in Figure 3.25(b). 
Figure 3.25(a) shows the two 'kissing faces' of the well- 
known face-vase illusion. 

One might object that the visual ambiguities in the 
figures are so artificial that they would never occur 
in the natural world, but that would not be accurate 
(Figure 3.26). In any room with rectangular walls, the 
corners make up parts of a Necker cube. (To demon¬ 
strate this, just roll up a piece of paper into a tube, and 
look at the corners of a room. You should be able to flip 
it visually from an inside corner to an outside one.) 

A cat stalking prey may look around a tree with only 
one eye, while the other eye just receives input from 
the tree: that leads to binocular rivalry, competition 



(a) (b) 


FIGURE 3.25 Ambiguous stimuli pose choices for interpreta¬ 
tion. Two famous ambiguous figures: (a) the "face-vase illusion' and 
(b) the Necker cube. The corners of a rectangular room will flip back 
and forth if they are seen through a tube that blocks out the visual 
surroundings. Source : Kim and Blake, 2005. 



FIGURE 3.26 An ambiguous object: What is it? Shapes are 
sometimes hard to identify, even under natural conditions, as you 
can tell by trying to see objects in a bush or wooded area, watching 
birds during a morning mist, or looking at oncoming cars when the 
sun is in your eyes. Source: Feldman, 2003. 


between the input to the two eyes. An animal walking 
through tall grass constantly receives different input to 
the two eyes, and some animals, like rabbits and deer, 
do not have overlapping visual fields at all, so that 
fused visual input is quite rare. 
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FIGURE 3.27 Which way is Humphrey Bogart looking? 
Human faces are among the most important objects in our envi¬ 
ronment. Being able to tell gaze direction and facial expression is a 
basic social skill. Source: Wexler, 2001. 


Faces are biologically important for humans and 
other primates. Mother-infant bonding occurs face 
to face in our species, for example, as do competi¬ 
tive confrontations, mating interactions, and the like. 
But faces also show ambiguities. Which way is 
Humphrey Bogart facing in the white-on-black photo 
(Figure 3.27)? What emotion does his face express? 
Which picture looks more skeptical or suspicious? 


3.6 Top-down or expectation-driven processing 

The brain constantly generates expectations about the 
world it encounters. Walking downstairs in the dark, we 
have expectations about every step we take. In dealing 
with ambiguities like the figures shown here, we con¬ 
stantly make predictions about which of two perceptual 
interpretations is the best one. Most words in English 
are ambiguous, so that even as you are reading this sen¬ 
tence you are resolving ambiguities. The brain is driven 
by more than just input; it has many ways of biasing 
choice-points by means of predictions and expectations. 
Lateral processing is also important, as we have seen, 
to sharpen differences between neighboring stimuli in 
the visual array. As we will see later, selective attention 
allows us dynamically to adjust our sensory biases (see 
Chapter 8), and long-term memory strengthens syn¬ 
apses that are associated with accurate perception. 

Maps and layers are not the only functional units. 
Many cortical regions are massively interconnected 
with each other, so that activity in one part of the 
cortex quickly spreads to other regions. A number 
of scientists believe, therefore, that the entire cor¬ 
tex, together with satellite regions like the thalamus, 
should be considered as a functional unit. This is often 
called the thalamo-cortical system. But to understand 
how those massive systems work, we will study their 
smaller components in some detail. 


FRONTIERS OF COGNITIVE NEUROSCIENCE 


The plastic brain 



FIGURE 3.28 Paul Fletcher, PhD, Behavioral and Clinical 
Neuroscience Institute, University of Cambridge, UK. 

For years, studies of learning in animals and humans 
have accounted for a large portion of experimental 


psychology research. It's easy to see why: A striking 
feature of animal behavior is the capacity to adapt - 
to learn which places and actions will bring pleasure, 
which will bring pain, and which are unreliable predic¬ 
tors of either. Added to this, the environment can be vari¬ 
able across time. A place or an action that was previously 
a reliable source of reward may rapidly become barren 
and the animal must be able to adapt to this, no matter 
how consistently rewarding that location previously has 
been. A key feature of the brain is its adaptability. 

But how are we to understand this adaptability in 
terms of the brain changes that must underpin it? The 
insights derived from neuropsychology alone, from the 
accidental occurrences that form the basis of lesion work, 
are not optimal because the processes that we are attempt¬ 
ing to scrutinize are dynamic and may well involve 
plasticity that is distributed across the brain rather than 
localized to a single region. In this regard, we can see the 
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conlrals jugglere juggler* 
scan 2 scan 2 scan S 


FIGURE 3.29 White matter changes 
occurring as a consequence of juggling training, 
(a) Colored voxels represent clusters (corrected 
p < 0.05) of significant FA increase from early 
to late training, superimposed on the mean 
FA map. (b) Mean FA change from scan 1 from 
within the cluster shown in (a). Error bars rep¬ 
resent standard errors, (^significant relative to 
baseline at p < 0.05; IPS = intraparietal sulcus, 
POS = parieto-occipital sulcus.) Source : Scholz 
et al, 2009. 


potential value of the remarkable developments in neu¬ 
roimaging that have occurred over the last two decades. 
Although much neuroimaging has been region-orien¬ 
tated, emphasizing functional segregation rather than 
functional integration, latterly there have been concerted 
attempts to understand learning, and the theoretical mod¬ 
els that describe learning, in terms of systemwide changes 
in brain structure and function. 

From a functional perspective, the brain is indubitably 
an ever-changing machine with regional activity modu¬ 
lated by experience. However, the change in localized 
regional activity is only a small part of the story and more 
recent approaches to functional neuroimaging have devel¬ 
oped ways of exploring interregional relationships, charac¬ 
terizing the learning process in terms of strengthening or 
weakening influences that different regions may exert over 
each other. A recent study has shown that the human brain 
is exquisitely sensitive to incidental contingency relation¬ 
ships between auditory and visual stimuli (den Ouden et 
ah, 2009). Intriguingly, though experimental participants in 
this study were largely unaware of these contingency rela¬ 
tionships, there was a measurable increase in the degree of 
connectivity between primary auditory and visual areas as 
a consequence of experiencing them, even though they had 
no bearing on the task that was being performed. Could 
such observations help to explain how extraordinarily able 
people are to function, under certain circumstances, as 
"intuitive statisticians" (see Shanks, 1995)? 

These functional connectivity measures are fascinat¬ 
ing but their structural underpinnings have been elu¬ 
sive. However, further recent developments in MRI have 
begun to allow us to determine how the key components 
of structural connectivity - the white matter tracts - may 


also change as a consequence of experience. Scholz and 
colleagues (2009) showed that learning to juggle was 
associated with changes in both gray matter density and 
in the white matter tracts (Figure 3.29). That is, as a con¬ 
sequence of simple practice in a motor skill, the brain is 
changing its structure in terms of gray matter density 
and its connectivity, in terms of white matter tracts. This 
is groundbreaking work; for many years there have 
been assumptions that the brain is structurally immu¬ 
table. Insights such as this point to both a functionally 
and structurally evolving organ, surely a step forward in 
refining our understanding of learning processes. 

Observations such as these at the macroanatomical 
level must surely complement more detailed observa¬ 
tions of synaptic change at the microanatomical level, 
and will provide the basis for a more complete under¬ 
standing of the brain's adaptability to changing contin¬ 
gencies in the environment. This understanding itself 
will be an absolute prerequisite for developing our 
understanding of the aberrant learning and connections 
that might characterize certain mental illnesses, most 
notably schizophrenia (Fletcher & Frith, 2009). 
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4*0 HOW NEURAL ARRAYS ADAPT 
AND LEARN 

Learning has been studied very intensively in layered 
arrays of simulated neurons. After a slow start, the 
study of 'neural nets' took off in the 1980s (Rumelhart 
and McClelland, 1986a, b). Connectionism has been the 
theoretical framework for much of this movement. 
Much of this chapter reflects lessons learned from the 
connectionist movement (see also Neural Darwinism, 
below). While neural nets were first explored in 
the 1950s, the early efforts encountered difficulties 
in doing useful processing. By the 1980s, it became 
clear that adding an additional (hidden) layer would 
solve many of those problems, and that feedback also 
helped. These insights helped to launch the connec¬ 
tionist movement. 


4*1 Hebbian learning: ‘Neurons that fire 
together, wire together* 

Donald Hebb (Figure 3.30) proposed, in 1949, that 
assemblies of spiking cells could learn an input pat¬ 
tern by strengthening the connections between cells 
that fire at the same time. This idea is encoded in the 
slogan that 'neurons that fire together, wire together'. 
It is a very useful learning method for neural net¬ 
works, and there is some direct evidence for Hebbian 
learning in the nervous system. 

The key idea in Hebbian learning is that more effi¬ 
cient synaptic connections are the physical substrate 
of learning and memory. As Figure 3.31 points out, 
however, there are a number of ways in which syn¬ 
aptic transmission could become more efficient. Two 
neighboring neurons can gain more synapses, the syn¬ 
apses may be supplied with more nutrients that lead 
to more neuro-transmitter chemicals, the receptors for 
those neuro-transmitters could become more efficient, 
and so on. 

Two kinds of synaptic changes are believed to 
be involved in learning; they can be considered 
strengthened excitation and strengthened inhibition. 
Long-term increases in excitation from one neuron to 
another are called long-term potentiation (LTP). Long¬ 
term decreases are called long-term depression (LTD). 
There is some evidence for both in the hippocampus. 

It is always important to remember how long the 
brain took to develop - the mammalian brain is some 
200 million years old and, prior to that time, even ear¬ 
lier vertebrate brains were able to develop high degrees 



FIGURE 3.30 Donald R. Hebb: cell assemblies that learn. 
Donald R. Hebb was one of the most influential theorists for cog¬ 
nitive science and neuroscience. He clarified the notion of a "cell 
assembly 7 , and proposed the best-known learning rule for neural 
networks, summarized by the slogan "neurons that fire together, 
wire together'. Source: Brown and Milner, 2003. 


of sophistication for their ecological niches. For that 
reason, while there is solid evidence for learning mech¬ 
anisms like Hebbian learning, it is believed that there 
may be other mechanisms as well. 

Hebbian learning can be shown visually as a thick¬ 
ening of lines between the nodes of a network, like the 
simple cell assemblies of Figure 3.32. In this case, a vis¬ 
ual circle is activating corresponding cells in a model 
of early visual cortex. The nodes in this circle represent 
points or small regions in the visual field, and they are 
being strengthened by correlated firing. That is the 
theory. Figure 3.33 shows a remarkable set of findings 
that seems to confirm the model, with actual strength¬ 
ening of connection probabilities ('weights') in the 
hippocampus of the cat. The brain is a large place, of 
course, so that finding apparent Hebbian learning in 
one location does not prove its existence everywhere. 
Nevertheless, this is an encouraging result, and sug¬ 
gests that the overall direction is productive. 
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FIGURE 3.31 Hebbian synapses and long-term potentiation. There may be several ways to increase the 
efficiency of synaptic connections. Source : Byrne in Squire et al., 2003. 





FIGURE 3.32 Hebbian learning in cell assemblies. Neurons 
are represented by circles and their connections by lines. Redder 
colors represent more active units and thicker lines indicate stronger 
connection weights between units. At time tl, the cell assembly 
encodes input in its connection weights. In this example, memory is 
retained at times t2 and t3. More realistic models may show forget¬ 
ting over time, and permanent memories may need to be strength¬ 
ened by repeated exposures to the same stimulus. Source : Abraham 
and Robins, 2005. 


In Figure 3.34 a simple network may be used for 
classifying input patterns. In this case, there are two- 
way connections vertically, with lateral inhibition in 
each layer of the network. This is sometimes called a 
TocaT network, because it does not involve learned 
strengthening of connection weights. It is nevertheless 


useful for explaining some basic cognitive phenomena 
(Rumelhart and McClelland, 1986b). 

Figure 3.35 shows a breakthrough in network mod¬ 
eling. Early models were devised in the 1950s by 
Rosenblatt (1962) and others. These were one or two- 
layer models, and were found to have logical limits 
when it came to basic brain functions like learning and 
recognizing patterns. For some time neural models 
went into decline, until some twenty years later, when 
it was realized that adding a layer of 'hidden units' 
and allowing the network to adjust its connection 
weights could solve the earlier difficulties. Figure 3.35 
shows a classical three-layer feedforward network 
with a hidden layer and adjustable weights. This net¬ 
work can learn efficiently when its output is compared 
to a wanted output pattern, and the network weights 
are systematically adjusted to come closer and closer to 
the set goal. This process is called 'back-propagation', 
and is much like a negative feedback loop, the kind 
that we find in thermostats. This kind of network 
is used most often today, though mathematical and 
empirical studies continue to advance the understand¬ 
ing of brain-like information processing. 

Figure 3.36 shows a self-organizing auto-association 
net, in which the output is led to match the input. 
That is a useful strategy for recognizing patterns like 
the sound of a familiar voice. Self-organizing systems 
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FIGURE 3.33 Observed Hebbian learning. Strengthening of neuronal connections 
has been observed directly in hippocampal neurons in the cat. Source : Sutherland and 
McNaughton, 2000. 



» . ■# Inhibitory connection 

-+ -► Excitatory connection 

FIGURE 3.34 A simple network. Notice the combination of exci¬ 
tatory and inhibitory connections. Source: Palmer-Brown et al., 2002. 


arise in nature in many situations. Biological organ¬ 
isms can be viewed as self-organizing systems, and 
perhaps nervous systems are a refinement based on 
biological cells. However, a marker of human culture 
is the immense amount of teaching that takes place. 
From birth onward, we learn from other people. Thus, 
culture may go a few steps beyond self-organizing 
learning systems, and develop knowledge and skills 
that can be taught, more like the network shown in 
Figures 3.35 and 3.36. 

Figure 3.37 shows how a self-organizing network 
may be applied to the fundamental human problem 
of face recognition. Babies learn to respond to normal 
faces (but not scrambled faces) early in their lives, and 
soon can tell familiar from unfamiliar ones. This exam¬ 
ple is much simpler than the task that babies solve so 
effortlessly, because it involves only a line drawing. 


Back-propagation network 


Input units 


Output units 



FIGURE 3.35 A classical three-layer network. The hidden layer 
makes the network much more flexible. Back-propagation allows net¬ 
work output to be compared to a teaching source, and changes net¬ 
work weights to match the source. Thus, the network can learn from 
feedback to approach its goal. Source : Abraham and Robins, 2005. 


The network learns to predict a mouth at the bottom 
of the picture, and two eyes on top. But it could not 
function under more subtle three-dimensional vis¬ 
ual conditions, in changing light conditions, moving 
mouths, eyes and heads, and so on. Nevertheless, the 
face recognition network in Figure 3.37 gives us a way 
to try to understand how the brain performs a basic 
life task (see Chapter 6). 

4.2 Neural Darwinism: survival of the fittest 
cells and synapses 

The neuroscientist Gerald Edelman has proposed that 
the brain is a massive selectionist organ. Edelman's 
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FIGURE 3.36 A pattern recognition net. An auto-associative network matches its output with its input, a feature that is 
useful for recognizing patterns, like faces or cars. On the right side some of the neurons have turned red, to show that their 
connection strengths have changed after learning. Source : Byrne in Squire et al., 2003. 
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FIGURE 3.37 A face recognition network. Recogni¬ 
zing faces is a basic problem for the visual system. With 
simple line drawings face-recognition can be done. But 
when real faces are seen under natural lighting conditions 
the job becomes far more complex and variable. Notice that 
is a three-layered network. Source : Luckman et al, 1995. 
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FIGURE 3.38 An example of Neural Darwinism 
in learning. This figure shows stages of encoding 
a neural activation pattern until dynamic synaptic 
activity allows permanent connections to be strength¬ 
ened, thereby enabling memories to be stored in the 
same locations where the original connections were 
made. Source : Ribeiro et ah, 2006. 


theory is called Neural Darwinism , since it suggests 
that neurons develop and make connections following 
Darwinian principles. In biological evolution, species 
adapt by reproduction , by mutations leading to diverse 
forms, and selection among the resulting repertoire of 
slightly different organisms. Over long stretches of time 
this weeding process yields species that are very well 
adapted to their niches. A similar process occurs in the 
immune system, where millions of immune cells adapt 
to invading toxins. Cells that can successfully match 
the invaders multiply, while unsuccessful ones dwin¬ 
dle in number. The immune system can therefore learn 
to recognize and combat even novel invaders from the 
environment. Selectionism therefore leads to very flexible 
adaptation. 

According to Edelman (1989), the brain has two 
stages of selectionist adaptation. The first begins soon 
after conception, when the first neurons are born, 
multiply, differentiate, and are selected if they fit their 
local niches. The outcome of this stage is a collection 
of neurons that looks like a brain. The second, over¬ 
lapping stage, begins when neuronal connections are 
made. Adaptive connections tend to survive while 
others die out. A kind of Darwinian selection there¬ 
fore operates both developmentally and as a result of 
learning (Figure 3.38). Two-way connections between 
neuronal maps allow for re-entrant (two-directional) 
processing. If a re-entrant process is stable, it will 
tend to survive among all the other transitory waves 
of activation in the brain. If not, it will simply fade 
away. 

If we add the Hebbian principle that 'neurons that 
fire together, wire together', a stable cell assembly will 
tend to make stronger connections between its units. 
Thus, the Darwinian process would lead to longer- 
lasting neuronal 'species', cell assemblies that enable 


the tasks of adaptation, learning, pattern recognition, 
and the like. 

These ideas have been refined by brain studies and 
computer simulations. The Darwin series of robots 
has employed selectionist principles to simulate how 
regions like the hippocampus seem to work. In Figure 
3.39a, a neural Darwinist 'rat' simulates the behavior 
of a rat in a Morris water maze, in which the animal 
needs to find a platform hidden under the surface 
of the water, which allows it to rest. In Figure 3.39b, 
a selectionist robot is learning to play soccer. While 
these tasks are very different, they are learned using 
the same principles. 

43 Symbolic processing and neural nets 

Standard computer programs use symbols. That is, 
they employ logical and mathematical expressions. 
Neural nets can be expressed in mathematical terms, 
but they tend to be more parallel (with many differ¬ 
ent computations happening at the same time), and 
distributed (able to process information in different 
places, using different memories, and so on). That is 
to say, neural nets are closer to biological information 
processing than to standard algebra and logic. 

However, humans created mathematics and logic, 
and natural language makes us very good at under¬ 
standing symbolic expressions. Most of the words we 
use are symbols referring to some class of events in 
the world. Furthermore, neural nets are easily trans¬ 
lated into mathematical expressions. 

How can these two different computational phi¬ 
losophies be reconciled? One method suggested by 
McClelland and Rogers (2003) is shown in Figure 3.40, 
where a neural network was designed to express 
symbols (like 'sunfish', and 'grow') and their logical 
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(a) (b) 

FIGURE 3.39 (a) This figure simulates a rat in a Morris water maze, swimming until it finds a platform to stand on. 

The brain-inspired simulation creates a neural map of the water maze, so that the simulation learns to find the platform. 

(b) A robot uses a Neural Darwinist brain model to learn to play soccer. Source: Neurosciences Institute, Krichmar, 2006, 
with permission. 



relations (like 'has a property'). For a human mind, 
this is a complicated way of expressing a simple idea 
like 'sunfish can grow'. But for massive neural net¬ 
works with many millions of units and trillions of 
connections - such as our brains - the McClelland and 
Rogers version is more compatible with the wiring of 
the brain. 

In sum, it seems that neural nets can be translated 
into symbolic form, and symbols can be converted into 
neural nets. Thus, the debate over symbolic versus 
neural net expressions is not either-or. The scientific 
question therefore would seem to be which theoreti¬ 
cal language is more useful for any given problem. If 
we want to understand brains at the level of neurons, 
networks are quite natural. If we want to understand 
the grammar of English, symbols might be better. It all 
depends on the question we want to explore. 

Adaptation and representation are two complemen¬ 
tary ways of thinking about the brain. We will see 
evidence that brains are specialized for knowing 
the world. There is also evidence that brains behave 
in adaptive ways, adjusting constantly to changing 


conditions. Both of these ideas seem to be funda¬ 
mental. Some findings may seem more adaptational, 
others more representational. We will find both per¬ 
spectives useful. 


5*0 COORDINATING NEURAL NETS 

The brain is often called a massively parallel organ, 
because neurons appear to be working quite inde¬ 
pendently of each other. There is no central command 
post that tells all neurons exactly what to do. 

There are a number of ways in which neurons can 
be coordinated, however. One way is for large-scale 
rhythms to pace populations of neurons, much like 
the conductor of a symphony orchestra. When many 
neurons fire in unison, their activity adds up, just as 
a large crowd of people sounds louder when they 
are chanting in unison. There is a limit to this, how¬ 
ever. Epileptic seizures have long been believed to be 
caused by neural scar tissue, called the epileptic focus. 
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FIGURE 3.40 Neural nets can handle symbolic 
expressions. In this example, a network represents a 
large set of propositions such as 'a robin is a bird' and 
'a rose has petals'. Source: McClelland and Rogers, 2003. 


which sends out intense, slow, and regular waves 
that recruit other brain regions, so that spreading 
populations of neurons begin to chant the same song. 
The result is a loss of consciousness and physical sei¬ 
zure activity. 

Obviously, the brain must balance the degree of 
pacing and coordination against the need for local 
neurons and their neighbors to work on local func¬ 
tions. There must be a balance between integration and 
differentiation (Edelman and Tononi, 2000). 

Figure 3.41 shows averaged correlations between 
visual regions while the subject was watching a movie. 
As you can see, similar regions in both hemispheres, 
symbolized by (1) and (r), tend to become active at the 
same time, as indicated by the red and yellow lines. 
Since both sides receive basically the same input from 
the two eyes, this result makes good sense. In addition. 


within each hemisphere strong correlations show up 
between early visual analysis and the area for visual 
object perception (LO). However, because this study 
used functional magnetic resonance imaging (fMRI) 
and averaged over long periods of time, we are not 
seeing fast correlated activity. This is a limitation of 
the specific methodology, which scientists are cur¬ 
rently working to overcome. A number of recent 
results show much faster gamma and theta correla¬ 
tion, for example, at the frequencies at which the brain 
seems to do much of its work (Fries, 2005). 

Regular EEG rhythms are now believed to signal 
distinct, coordinated processes. For example, a high 
density of gamma rhythms has been related to con¬ 
scious visual perception, and to the process of dis¬ 
covering a solution to a simple word problem. Alpha 
rhythms are traditionally associated with an absence 
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(a) (b) 


FIGURE 3.41 Correlated brain regions while watching a movie. A dramatic 
illustration of correlated activation in the visual cortex while the subject was watch¬ 
ing a movie. The upper half of the large panel is in the left hemisphere, the lower 
half in the right hemisphere. V areas with the same number correspond to the left 
and right sides, which are usually connected across the midline. "Hotter colors' and 
thicker lines indicate higher correlations between visual areas. Notice that the strong¬ 
est correlations are between corresponding regions of left and right hemispheres. 
Lop = posterior part of the lateral occipital complex; LOI = lateral part of the lateral 
occipital complex; (1) = left hemisphere; (r) = right hemisphere. On the left are local 
fMRI activations. Source: Bartels and Zeki, 2005. 



FIGURE 3.42 A pattern-recognition network. The yellow star in the upper left panel provides an input shape into a 
pre-trained neural network. Later panels give the current state of the network, shown in an array of colored dots. Notice 
that the yellow star emerges slowly, as the network recognizes the input, and eliminates alternative possibilities. Source: 
M. Shanahan, Imperial College London, with kind permission. 
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of focused attentional tasks, but theta rhythms are 
now believed to coordinate the hippocampal region 
and the frontal cortex during the retrieval of memo¬ 
ries. And delta rhythms, the traditional signal of deep 
sleep, are believed to group fast neuronal activity in 
order to facilitate the consolidation of learned events 
(Kemp et al. r 2004). 

Figure 3.42 shows a very simple and reasonable 
hypothesis about how regular brain rhythms may 
coordinate the firing of millions of separate cells. 
Neurons that fire at the peak of the alpha wave (for 
example) add a tiny amount of electrochemical 
activity to the whole chorus. Neurons that fire during 
the trough of the regular rhythm subtract their activ¬ 
ity. Thus, neurons that fire in sync with the dominant 
rhythm are strengthened by feedback from the mil¬ 
lions of other neurons that are keeping up the alpha 
rhythm, while those that are out of sync are weakened. 
Such a mechanism would tend to reinforce rhythmic 
firing. 

As we have already pointed out, however, simple 
synchronized firing is not enough. Different coali¬ 
tions of firing neurons must be able to generate differ¬ 
ent input representations, which can compete against 
other coalitions to recruit new members. Such a model 
is shown in Figure 3.38, where it supports a kind of 
Neural Darwinian competition between different pop¬ 
ulations of nerve cells. 


5*1 Functional redundancy 

When engineers build airplanes, they always introduce 
functional redundancy into their designs, so that there is 
a backup for critical functions that could go wrong. 
If one jet engine fails, most aircraft are designed to 
fly using the remaining ones. Humans and animals 
also evolved with functional redundancy in all their 
organ systems - we have two lungs, two sides of the 
heart, and so on. The brain is no exception. Even the 
loss of the speaking half of cortex can be overcome, if 
it occurs early in childhood. The brain can often keep 
working, even in the face of some damage. 


6*0 SUMMARY 

The most basic question in cognitive neuroscience is 
how nerve cells can combine to perform complex cog¬ 
nitive functions, like perception, memory, and action. 
That problem has not been solved in all its richness. 


but significant progress has been made. The integrate- 
and-fire neuron has been well studied, and is under¬ 
stood in considerable detail. However, there are a 
great many kinds of neurons, types of transmission 
between them, and neurochemicals that play cru¬ 
cial roles. We take the integrate-and-fire neuron to be 
our prototype, because it is well established and can 
be simplified, so that its action can be understood by 
detailed modeling. Both artificial neural networks 
(ANN) and biologically inspired networks are useful 
for this purpose. 

Neurons sometimes make up simple circuits, like 
the knee-jerk reflex. More often, however, they organ¬ 
ize themselves into large two-dimensional arrays, 
which link higher and lower level arrays into hier¬ 
archies. All the sensory and motor systems can be 
viewed as such complex hierarchies. In vision, touch, 
and motor control, arrays of neurons are topographi¬ 
cally arranged as 'maps' of the spatial surroundings. 
The visual system, for example, can be viewed as a 
hierarchy of topographical maps. 

Hierarchies are not rigid, one-way pathways. They 
allow signals to flow upward, downward, and laterally. 
A major function of downward flow of information in 
the sensory systems is the need to resolve ambiguities 
in the input. Ambiguities are common in visual scenes, 
but also in language and the other senses. In motor 
systems, upward (bottom-up) flow of information is 
similarly useful to help make choices in output plans 
and motor commands. 

Lateral inhibition is a widely used biological strat¬ 
egy for emphasizing differences between inputs, like 
two patches of light and dark in a visual scene. Cells 
in the sensory systems have receptive fields that are 
attuned to specific types of input, such as line orien¬ 
tation, color, movement, shape, and object identity. 
As the visual maps go deeper into cortex, their spa¬ 
tial resolution becomes less, but their ability to inte¬ 
grate large amounts of information goes up. While 
we study the sensory and motor systems separately, 
the brain is a giant sensorimotor engine, which 
allows constant interaction of input and output at all 
higher levels. 

Spatial arrays of neurons support spatial coding of 
information, but there is temporal coding as well. The 
major regular rhythms of EEG are believed to corre¬ 
spond to massive coordinated activities among large 
populations of neurons. Recent research suggests that 
gamma activity may be involved in such processes as 
sensory integration into conscious percepts, and theta 
has been associated with retrieval from long-term 
memory. 
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7*0 STUDY QUESTIONS AND 
DRAWING EXERCISES 

7.1 Study questions 

1 Describe the basic function of an integrate-and- 
fire neuron. 

2 What is lateral inhibition and how does it relate 
to perception? 

3 Explain how sensory and motor regions can be 
viewed as hierarchies. 

4 Describe the role that re-entrant (two-way) con¬ 
nections play in brain function. 

5 What is Neural Darwinism and what aspects of 
brain processes does it relate to? 

6 List three or more common features of sensory 
systems. 

7.2 Drawing exercises 

1 Label the parts in Figure 3.43. 

2 Draw Hebbian learning of cell assemblies for 
a visual circle. What slogan describes Hebbian 
learning? 




FIGURE 3.43 An idealized neuron. A simplified neuron 
for you to label. 
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I believe that the study of neuroimaging has supported localization of mental operations within 
the human brain . 


Michael L Posner (2003) 



Different views of the "internal highway system' of the cortex - fiber tracts running 
through the inner brain, using a method called Diffusion Tractography. All fibers in 
these images are running crosswise across the corpus callosum, and then arch upward 
like fountains on either side. (A) is a side view, showing fibers sweeping upward in the 
left side of the hemisphere. Green fibers are in front (prefrontal), light blue begin and 
end in premotor cortex, dark blue fibers are coming from the motor cortex itself, orange 
ones are parietal, and yellow fibers flow from the occipital cortex on one side of the 
brain to the other. The violet fibers along the side run between the temporal lobes on 
each side of the brain. (B) shows just the fiber bundles alone, on both sides of the brain, 
viewed from back (yellow) to front (green). On the bottom right, note the orientation 
compass: A, anterior; I, inferior; L, left; P, posterior; R, right; S, superior. Source: Hofer 
and Frahm, 2006. 
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1.0 INTRODUCTION 

This chapter presents the tools of brain recording from 
the viewpoint of an intelligent user, trying to under¬ 
stand how the brain works. The Appendix presents 
the tools in more detail. 

A perfect observer of the mind-brain would be able 
to follow tens of billions of neurons and sample each 
one a thousand times per second. The perfect observer 
should then be able to track the constantly shifting inter¬ 
play between smaller and larger groups of neurons, 
making trillions of possible connections. By analogy, a 
perfect spy satellite in space would be able to see every 
single human being, as well as the changing relationships 


between individuals and groups, from families to whole 
nations. 

Such a perfect observer does not exist. Our under¬ 
standing of the brain is a kind of collage of many 
fragments of the puzzle, glued together to make a rea¬ 
sonable picture of the whole. But thinking about a per¬ 
fect observer gives us the parameters we can aim for. 

Brain imaging has been a breakthrough technology 
for cognitive neuroscience, building on decades of cogni¬ 
tive psychology, behavioral conditioning, psychophysics 
and brain science. Before imaging techniques matured, 
our knowledge came from animal studies and the hap¬ 
hazard injuries incurred by human beings. But brain 
injuries are extremely imprecise, and even to locate the 
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damage, neurologists often had to rely on post-mortem 
examination of the patients' brains - as in the case of 
Broca's and Wernicke's patients discussed in Chapter 
1. The brain can often compensate for damage, so that 
lesions change over time, as cells die and adaptation 
occurs. Therefore, post-mortem examinations do not nec¬ 
essarily reflect the injury at the time of diagnosis. Animal 
studies depend on presumed homologies - similarities 
across species - that were often not convincing to eve¬ 
rybody. No other animals have language, and other dis¬ 
tinctively human specializations. It was therefore very 
difficult to understand how brain functions in animals 
mapped onto human cognition. 

Many of these problems were resolved when it 
became possible to observe the living brain, first by 
electroencephalography (EEG), then by X-rays, then 
computer tomography (CT, the study of slices - from the 
Greek word for 'slice', tomos ) based on X-rays, positron 
emission tomography (PET) and magnetic resonance 
imaging (MRI, etc.). Today, we have perhaps a dozen 
techniques that are rapidly becoming more precise. 
Medical needs often drive this very expensive technol¬ 
ogy because it applies to many organs in the body. As 
a result, we now have ways to study the distribution 
of billions of neurochemical receptors in the brain, the 
thickness of cortex, the great highway system of white 
fiber bundles, and most important for cognitive neu¬ 
roscience, the functional activity of the brain - the basis 
of its adaptive capacities. New advances are allowing 
scientists to investigate not only functional activity 
located in specific brain regions, but also to measure 
the dynamic pattern of connectivity between them. 
Some of the massive 'wiring' of the brain is shown in 
the figure at the beginning of the chapter, but like the 
world wide web, the wiring is only part of the story: 
there are ever-changing dynamic connections made 
between neural populations that can alter in a fraction 
of a second. 


1.1 Brain recording: more and less direct 
measurements 

Some neuroimaging methods pick up neuronal activity 
more directly than others. If we place an electrode next 
to (or inside) a neuron we can record its electrical activ¬ 
ity, typically its axonal spikes. This single-electrode 
measurement can be seen as a direct measure of the 
electrical activity of the neuron. As we have seen in 
Chapter 3, however, neurons do more than fire spikes. 
The input branches of a neuron, the dendrites, also 
engage in important activity. By recording different 


parts of a neuron we get somewhat different measures 
of its activities. 

This is nicely illustrated in a recent study by 
Quiroga and colleagues at Caltech (Quiroga et al. f 
2005) (Figure 4.1). These researchers found that a sin¬ 
gle neuron in the medial temporal cortex was selec¬ 
tively activated by pictures of actress Jennifer Aniston. 
It was not activated by pictures of other actresses, or 
actors, or by other kinds of images such as houses, 
scenery, and objects. The researchers also found other 
cells that were highly selectively activated, includ¬ 
ing a neuron that was sensitive to pictures of actress 
Halle Berry and one for the Sydney Opera House. It 
is important to keep in mind, of course, that we are 
sampling just some out of billions of neurons. In all 
probability, there is a complex network of related neu¬ 
rons, and a lucky hit will reveal a Jennifer Aniston or 
Sydney Opera House fan. 

The researchers used the axonal firing rate of single 
neurons as the measure of their activity. Would this 
mean that this single neuron 'knows' about Jennifer 
Aniston? Not at all. We should rather think of the 
activity of this cell as a representative for a large and 
widely distributed network of neurons that are sen¬ 
sitive to one particular face. If this neuron were lost, 
the person would still be able to recognize Jennifer 
Aniston. The brain as a whole would not show a 
detectable change. 

Single cell recordings are only rarely possible in 
human subjects. It is ethically allowable in patients 
with medically unbeatable epilepsy or brain tumor, 
where the only treatment is surgical removal of the 
affected region. Depth electrodes are inserted to iden¬ 
tify vital cognitive areas such as the language areas. If 
such a surgical procedure is required, scientists may 
be allowed a brief time to test a subject while the elec¬ 
trodes are in place. 


1.2 The time-space tradeoff 

Today's most popular methods are shown in Figure 4.2. 
Notice that they do not yet have the space or time res¬ 
olution needed to track single neurons, or even small 
clusters of neurons, like the columns of the cortex. 
They are gaining ground, however. Techniques like 
fMRI, which record physiological changes like blood 
oxygenation, are typically thought to have good spa¬ 
tial resolution and relatively poor temporal resolution. 
fMRI has a response time of about six seconds, because 
changes in local blood supply take some time - too 
slow for tracking neurons and neuron populations 
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FIGURE 4.1 The Jennifer Aniston neuron. Quiroga and colleagues (2005) found a neuron in the left hippocampus that selec¬ 
tively responded to different views of the actress Jennifer Aniston. (a) Responses in 30 of a total of 87 images are shown, (b) Axonal 
spikes to 30 out of 87 photos are shown. Numbers indicate the image number; graph indicates number of neural spikes recorded. 
Single neurons are likely to represent large networks with different sensitivities and preferences. 





directly. However, some recent studies show fMRI sig¬ 
nal reflects neuronal firing six seconds before. 

fMRI has very good spatial specificity compared 
to EEG and magnetoencephalography (MEG), which 
use electrical and magnetic signals respectively. Thus, 
fMRI is often used to localize brain functions. But EEG 
and MEG have excellent temporal resolution - almost 


instantaneous - and relatively poor spatial precision. 
They can track cell populations firing and fading over 
tens and hundreds of milliseconds, but it is hard to 
know which set of neurons is causing the signal. Some 
studies therefore combine EEG with fMRI to obtain 
the best temporal and spatial precision. A great deal 
has been learned with combined recording techniques. 
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PET Lesion 


Microlesions 


— — - » 

Millisecond Second Minute Hour Day 

Log time 

FIGURE 4.2 Pros and cons of imaging techniques. Different imaging modalities have different resolution. 
While some approaches have a very high temporal resolution but a low spatial resolution, other modalities 
have an opposite relation. 


Here we present some MRI images to demonstrate 
the ability they provide to map out brain regions. 
(Figure 4.3). Differing views can be taken on brain 
anatomy using different slices, such as coronal (Figure 
4.4), horizontal, or midsagittal (Figure 4.3), and recon¬ 
structed into a three-dimensional image (Figure 4.4). 

2.0 A RANGE OF USEFUL TOOLS - 
MEASURING ELECTRIC AND 
MAGNETIC SIGNALS 

2.1 Single-unit recording 

Our most precise recording method today is single¬ 
neuron or 'unit' recording, using deep needle elec¬ 
trodes, sometimes implanted in the brain (Figure 4.5). 
Unit recording is often done so as to sample a dozen or 
even a few hundred neurons in a single study. Needle 
electrodes typically pick up axonal spikes, which 
are believed to be crucial in information processing 
and transmission. But unit recording has a sampling 
problem - how do we know that local spikes represent 
a whole region of the brain? It also matters whether 
cells are excitatory (making others fire more) or inhibitory 


(making others fire less). In addition, inserting a needle 
into the brain is invasive and potentially harmful - 
it requires surgery. It is therefore done only in experi¬ 
mental animals or in humans with medical conditions 
such as unbeatable epilepsy, where exploratory sur¬ 
gery is required. We will see examples of each kind. 

We can, in fact, record from single neurons that do 
important things, ever since Hubei and Wiesel (1962) 
were able to record single feature-sensitive cells in the 
visual cortex of the cat, an achievement for which they 
received a Nobel Prize in 1981. More recent work has 
recorded in medial temporal lobes (Figure 4.6). Like 
every method, electrical recording of axonal firing has 
its limitations, but it continues to be a major source 
of information. Neurons fire a maximum of 1000 Hz, 
but cortical neurons average about 10 Hz. We have 
no 'universal census' of cortical neurons, so we do 
not know with certainty how representative the small 
samples we can observe really are. 

In many countries deep electrode recordings are 
allowed in primates, such as the macaque monkey, under 
suitable ethical constraints. The macaque brain has some 
striking similarities to human brains. Single-neuron 
recording in the macaque prefrontal cortex may show 
working memory phenomena. In a typical experiment, a 
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FIGURE 4.3 A brain navigation program. Brain navigation software allows the user to translate precise 
locations (in x, y, z coordinates) into brain locations. Notice the orientation of the standard slices. The x, y, z 
coordinates are known as the Talairach system (Talairach and Tournoux, 1988). Anatomical landmarks like the 
corpus callosum can be seen in the upper left display. Source : Mai and Thomas, 2006, with kind permission. 



FIGURE 4.4 A coronal 'sausage slice'. A close-up in three 
dimensions from the Brain Navigator software. Yellow and Red 
colored shading show medial temporal regions in the left and right 
hemispheres. Source: Mai & Thomas, 2006, with permission. 

macaque is trained to fixate visually on a cross on a com¬ 
puter screen, and to perform a delayed response to a vis¬ 
ual stimulus. The animal is trained to wait for a moment 
before looking in the direction where a stimulus appears; 
or alternatively, to look in the opposite direction. We can 



FIGURE 4.5 Single neuron recording deep in the brain. 
Intracellular and extracellular recording of single cells. Notice also 
the ability to stimulate single neurons, using the same electrodes. 
Single unit recording comes close to the desirable level of tempo¬ 
ral and spatial resolution for brain recordings. It works very well in 
tracing circuits of neurons. However, this method does not allow us 
to track large-scale populations of neurons. 
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FIGURE 4.6 Unit recording in the hippo¬ 
campus. Microscopic needles can be inserted into 
single neurons, or they can record extracellular 
electrical field potentials. The same electrodes 
can then be used to stimulate specific cells. The 
hippocampus contains regular arrays of neurons 
with distinct functions. Source : Squire et al., 2003. 


then record the activity of a single prefrontal neuron in 
the three phases: (1) the presentation of a visual stimu¬ 
lus; (2) the period when the monkey keeps the location 
of the visual stimulus in working memory; and (3) the 
response of looking in the direction of the stimulus after 
it disappears, or in the opposite direction. This is illus¬ 
trated in Figure 4.7. 

Depth electrodes have been used in humans. Typically, 
these electrodes are implanted before surgery in a patient 
who has otherwise unbeatable epilepsy. The implants 
can determine where epileptic foci begin at the onset 
of a seizure, and where critical regions of the brain are 
located that must not be lesioned (Figure 4.8). In addition 
to localizing epileptic foci, depth electrodes have been 
used to assess areas in the brain that decode information 
such as sematic relatedness, shown in Figure 4.9. 

How does recording in a single neuron relate to 
human perception? While a single cell cannot tell us 
much about human cognition, a recent experiment pro¬ 
vided some intriguing results regarding conscious and 
unconscious visual perception (Figure 4.10). 

While the spiking neuron is a plausible unit of brain 
activity, there are important alternative views. Some 
scientists believe that graded dendritic currents in 
each neuron may do useful information processing; 
some argue for subcellular processes inside the cell; 
others point to non-classical cells and synapses, which 


are known to be much more common than previously 
thought; others believe that glial cells participate in 
information processing; and many scientists believe that 
real brain processes only take place at the level of popu¬ 
lations of neurons. Therefore, recording axonal spikes is 
important, but it may not be the only important thing 
going on. Obviously, it's a risky business to jump from 
a single neuron to more than 10 billion in the vast forest 
of the brain. 


2.2 Animal and human studies cast light on 
each other 

Non-human primates, such as macaque monkeys, have 
been extensively studied using single and multiple 
unit recordings. Much of what we know about vision, 
memory, attention, and executive functions comes from 
studies of the macaque (Figure 4.11). 

How do brain regions in macaque correspond to 
human brain regions? While there are clearly some major 
anatomical differences, especially in frontal and pari¬ 
etal lobe regions, there remain some strong similarities 
between macaque and human brains (Figure 4.12). 

Single unit studies in macaque do not just map sen¬ 
sory and perceptual features, they may also involve 
more cognitive aspects such as attention. 
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FIGURE 4*7 Working memory activity in single neurons. The macaque monkey is perform¬ 
ing a working memory task called Delayed Match to Sample (DMTS). The "sample stimulus' in 
this case is the blue dot in the display (a). Note that the monkey successfully presses the blue 
disk about 20 seconds later. Thus the animal can "match to sample" even after the original stimu¬ 
lus is gone, implying that it must be kept in memory for a brief period of time, (b) shows a raster 
scan with neuronal spikes for each trial. Each horizontal set of small black lines is one trial. We 
can see that the neuron is firing faster in the Memory Period (Delay) compared to the preceding 
and following seconds, (c) shows a histogram of the firing of the neuron, located in the temporal 
lobe. Notice that the red columns in the Delay period indicate that the "memory neuron" keeps 
firing even when the stimulus is not shown, but is held in immediate memory. This particular 
neuron has a background firing rate shown in blue, of 5-10 Hz, typical for cortical cells. However, 
between the sample stimulus and the successful matching choice, the neuron doubles its firing 
rate, shown in red. Such neurons are believed to be involved in the temporary stage of Working 
Memory contents. They often occur in the temporal and prefrontal lobes. Source: Fuster, 1997. 


Single unit studies have provided us with a wealth of 
information regarding the encoding properties of neu¬ 
rons across cortical regions. However, neurons respond 
as ensembles, with complex networks of cells in multi¬ 
ple locations. How can we capture this type of dynamic 
brain response? An early technique for doing just that is 
presented in the next section: electroencephalography. 


2.3 Electroencephalography (EEG) 

The brain's large-scale electrical activity can be recorded 
through the scalp or on the surface of the cortex. Rather 
than picking up electrical activity direct from neurons, 
which are essentially tiny batteries, the electroencepha¬ 
logram picks up the electrical field. The resulting brain 
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(a) (b) (c) 


FIGURE 4*8 Depth electrodes in humans. While most single-cell recording is done in animals, human studies 
have been done when depth electrode recording is medically necessary. The arrows point to electrode placements 
in the temporal lobe. If you look carefully at the left MRI scan (a), you can see the electrode tracks, and the small 
holes in the rear of the scalp through which they were inserted. Neurosurgery like this is generally safe and pain- 
free, because the brain itself does not contain pain-sensing neurons. Source: Diet! et al., 2005. 



Time (sec) 


FIGURE 4.9 An example of the results from electrocorticographic 
(ECoG) recordings showing high-gamma oscillation (ERS - event- 
related Synchronization) which have been hypothesized to form the 
brain bases for 'binding' of information across sensory and cognitive 
domains. This figure shows the results of ECoG recordings during a 
lexical semantic processing task (deciding if two words were semanti¬ 
cally related, such as 'knife' and 'fork'). Recording sites are shown color 
coded into white, yellow, and blue. Blue colored sites showed activa¬ 
tion during auditory word stimuli but not during visual word stimuli, 
and so the authors described these sites as modality (auditory) specific. 
Further, these sites showed no modulation by semantic relatedness, 
which is shown in the lower two blue inset boxes. Yellow colored sites 
showed activation to both visual and auditory word stimuli, and hence 
the authors described these sites as modality-independent. These sites 
did show an effect of semantic relatedness, which is shown in the upper 
two blue inset boxes. Source : Crone, Sinai, & Korzeniewska, 2006. 


record is referred to as an electroencephalogram (EEG) 
or electrical brain record. The EEG was discovered in 
1929 by Hans Berger (Figure 4.15). Because electromag¬ 
netic waves propagate essentially instantaneously, EEG 
is highly sensitive temporally. However, EEG is quite 
selective, being more sensitive to neurons near the sur¬ 
face than to deeper neurons. EEG is picked up through 
layers of moist tissue, so that the original electrical 
activity is attenuated and distorted by the shape and 
conductive properties of the intervening cells. Some 
researchers believe that EEG is largely sensitive to the 
first layer of the cortex, which mainly consists of a tightly 
woven 'feltwork' of cortical dendrites (Freeman, 2004). 

It is easiest to record EEG from the scalp, though it is 
sometimes recorded from electrodes placed directly on 
the cortical surface. Like unit electrodes, EEG is a rela¬ 
tively direct measure of the brain's electrical activity. 
But with tens of billions of cortical neurons firing about 
10 Hz, we have several trillion electrical events per sec¬ 
ond. The raw EEG is therefore difficult to understand, 
and it was difficult to interpret before the advent of 
powerful computerized analysis. 

However, when the EEG is averaged over a number 
of experimental trials and 'locked' to a specific zero 
point, like the onset of a stimulus, the averaged elec¬ 
trical activity yields elegant and regular waveforms. 
This event-related potential (ERP) is sensitive to large 
neuronal population activity that characterizes visual, 
auditory, and even semantic processes. 

2.3.1 Sampling populations of neurons 

The activity of large-scale populations and networks 
is another important level of analysis (Freeman, 2004; 
John, 2004). Spontaneous EEG shows different patterns 
of activation. The brain can operate with many different 
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FIGURE 4.10 Human single cell recording and conscious perception. A remarkable experiment in which both 
conscious and unconscious stimuli were shown simultaneously to the two different eyes, using a variant of binocu¬ 
lar rivalry (see Chapter 6). In the upper row (c), the woman's face is conscious but the ball is not; in the lower row 
(d), this is reversed. Peak firing rates, in red, during time periods marked by green horizontal bars, when the sub¬ 
ject is indicating that the woman's face can be seen. The brain seems to determine which of the two simultaneous 
stimuli will become conscious when the signal reaches object recognition cortex. The electrode locations are shown 
on the brain scans on the left. Source : Rees et al., 2002. 



FIGURE 4.11 Monkeys have striking similarities to humans. 
Macaque monkeys are extensively studied because of apparent 
brain homologies (biological similarities) to humans. The macaque 
visual brain has been our best guide to the human visual cortex 
until very recently, when it became possible to study the human 
cortex more directly. Macaques also have emotional similarities to 
humans, close infant-mother bonding, and even prefrontal regions 
that resemble the human prefrontal cortex. Obviously, they do not 
have language and other species-specific human traits. Source: PLOS 
Biology open access journal. 


levels of interactivity. For example, during deep sleep, 
the raw EEG shows large, slow waves. This indicates 
that large groups of neurons are synchronized on a 
very large scale throughout the brain. When the subject 
wakes up, this slow pattern is replaced by small, rapid 
electrical waves, indicative of rapid and flexible infor¬ 
mation patterns of interaction in the brain. It is currently 



FIGURE 4.12 Brain homologies between humans and 
macaques. Above the human brain, and below, the macaque brain. 
The yellow areas are specialized in visual object recognition in both 
species. Both are shown from the right lateral perspective, looking 
to the right, (p) = posterior, (a) = anterior. The ventral view shows 
the bottom of the right hemispheres. Source : Squire et ah, 2003. 
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BOX 4.1 ECoG: Direct brain recording 


Even with fast-improving brain imaging techniques, the 
most direct evidence about the living brain still comes from 
intracranial electrical recordings. One reason is that the elec¬ 
trical signal strength in the brain is much greater than on 
the scalp - on the order of hundreds of millivolts rather than 
microvolts. Surface EEG is also filtered through a watery 
medium of brain tissue, skin, and muscle, the last with its 
own electrical activity. For example, when you frown, the 
muscles above your eyes contract, and along with it thin 
layers of muscle across the scalp stretch and adjust. Even 
eye movements have large effects on the scalp-recorded 
EEG. Thus, surface EEG recordings mix many electrical 
sources, as well as being filtered by layers of tissue. ECoG 
(electrocorticography) has an inherent advantage in signal- 
to-noise ratio. The biggest drawback is that ECoG requires 
invasive surgery. It is therefore never done without clear 
medical justification in humans. Numerous animal studies 



use ECoG or depth electrodes and still provide much of the 
basic evidence about brain activity. 

Wilder Penfield and his colleagues pioneered electro¬ 
corticography in humans in the 1950s. Epileptics with 
uncontrolled seizures can benefit from surgical removal 
of seizure-causing patches of the brain. ECoG recording 
can show where such "epileptogenic foci" are located. In 
addition, the surgeon needs to know which areas to avoid 
cutting, because they are vital for perception and lan¬ 
guage. ECoG studies are relatively safe and practical prior 
to surgery. Probing the cortical surface is generally pain- 
free, because the cortex does not have pain receptors. The 
scientific benefits have been very important (see Chapter 
8). While animal studies have laid the foundations for 
direct brain recording, ECoG studies in conscious humans 
are helping to uncover the neural basis of language, 
conscious perception, and voluntary control. 


FIGURE 4.13 (a) A surgically exposed cortex with an 8 X 8 

electrode grid placed directly on the surface. The electrodes (white 
circles) are mounted on a transparent plastic square that is over¬ 
laid on the left hemisphere. Notice that the grid covers part of the 
temporal lobe, a common source of seizure activity. It also samples 
the receptive language region of the left hemisphere (Wernicke's 
area), as well as Broca's area (the left inferior frontal gyrus), which 
controls language production. The left hemisphere is dominant for 
language in most people, and the electrodes are therefore placed 
over the most critical locations, (b) An X-ray radiograph of the 
same patient, superimposed on a structural MRI of the lateral 
brain. Source : Dalai et al. (2008). 


(Continued) 
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BOX 4.1 (Continued) 
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FIGURE 4.14 The results of an ECoG study of an epileptic patient, shown over an MRI of the lateral 
brain. The three inset graphs show a fast increase in high gamma activity (80-100 Hz) immediately after a 
picture with an object to be named is presented. The patient is asked to name the object, and the electrical 
activity is averaged over all presentations. The colored strips show which electrode pairs were used for 
cortical stimulation, which typically interferes with the functioning of the underlying cortical area. The red 
and purple strips mark places where electrical stimulation interfered with picture naming, mouth move¬ 
ments, and other language responses. The green strips indicate electrode pairs for which electrical stimula¬ 
tion caused no task interference. Source: Crone, Sinai, & Korzeniewska, 2006. 


,VVAA/VN/\AAA^\A^A/VVV\AA/VVVV\AA/N/u^A/VVV\AA/NA/VV'vVVVV l vVVVyA/ v ^V\ 

FIGURE 4.15 The first EEG. This historic EEG signal was 
recorded by Hans Berger in 1929, using a single pair of electrodes, on 
the scalp. Above, an EEG signal recorded on Berger's son, showing 
alpha activity (the regular sine wave in the upper trace). The lower 
trace is an electrical timing wave of 10 Hz. Source: Gottesmann, 1999. 

believed that cortical neurons do not fire at different 
average rates during deep sleep compared to waking, 
but rather that waking EEG allows a far greater amount 
of interactive processing. Sleep is not a passive state, 
therefore, but a different operating mode of the brain, 
with its own functions. One of these is believed to be 
the consolidation of memories based on waking experi¬ 
ences (Hobson and Stickgold, 1995). 

The standard method for analyzing highly com¬ 
plex EEG is Fourier analysis, named after the French 
mathematician Pierre Fourier, who showed that any 


complex signal can be decomposed into sine waves 
with differing amounts of power (density) in each fre¬ 
quency range (Figures 4.16). 

The 'raw' (unprocessed) EEG shows visibly dif¬ 
ferent waveforms like alpha, beta-gamma, theta and 
delta. However, raw EEG is believed to be a combina¬ 
tion of many different kinds of activity. Using mathe¬ 
matical analysis, we can decompose these complex and 
noisy waveforms into frequencies components. This 
is not unlike taking noisy radio static and decompos¬ 
ing it into frequency bands - which is exactly what we 
do by tuning to a specific radio station. The analyzed 
EEG shows that certain tasks may show a specific 
rhythm, such as alpha or gamma, in specific regions 
of the brain. This is illustrated in Figures 4.16 and 4.17. 
Some of these typical frequency bands are shown in 
Table 4.1. 

The classical EEG waveforms can be picked up directly, 
from the 'raw' or unanalyzed EEG. Notice especially the 
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FIGURE 4.16 Regular rhythms in different parts of the brain. A method called Fourier analysis allows us to decompose the density 
(or power) of regular wave forms that are buried in noisy EEG (see Table 4.1). The graphs show the resulting power curves. According to 
this source, the greatest alpha density is found over the occipital cortex, while the greatest theta density is over the frontal cortex. Theta 
is thought to involve hippocampal-frontal interactions during long-term memory retrieval. Gamma is found widely through the brain, 
and is believed to reflect functional interactions between different regions during the conscious state. The colors correspond to different 
frequency ranges. Source : From Zoran Josipovich, with permission. 


widespread, low-amplitude beta and gamma activity typ¬ 
ical of waking conscious states. The opposite state occurs 
in deep sleep, when the EEG is slow, high in amplitude, 
and much more regular. Other unconscious states show 
similarities (Baars et ah, 2004). Alpha activity may be seen 
over the occipital cortex when subjects are in a relaxed but 
alert state. 

As mentioned, while EEG has a millisecond time 
resolution, its spatial resolution is rather poor. It is very 
difficult to locate the electrical source of the EEG signal. 
It helps to increase the number of electrodes and to use 
sophisticated analytical methods. However, EEG gives 
us little information about brain regions deep beneath 
the cortex. Since subcortical regions like the thalamus 
are very important, EEG seems to have inherent limits. 

Early uses of EEG were to evaluate brain responses 
during sleep and dreaming states (Figure 4.15). New 
improvements using high-density arrays of electrodes 
have enabled scientists to make better, more detailed 
measurements of brain responses. 


An important aspect of EEG recording is to measure 
the evoked potential (ERP) that occurs in response to 
specific stimuli. To improve the signal to noise, these 
responses are averaged over many trials. 

For example, visual ERPs have been measured to 
investigate the time course of explicit memory processing 
(Figure 4.18). The ERP is sensitive to many factors that are 
important in human cognition, such as hearing your own 
name or listening to music (Figure 4.19). EEG reveals 
brain patterns during sleep and waking, abnormalities 
during diseases like epilepsy, and even the brain areas 
that respond to music. A more recent technique, magne¬ 
toencephalography (MEG), is highly related to EEG and 
has provided new ways to image the human brain. 

2*4 Magnetoencephalography (MEG) 

Magnetoencephalography (MEG) measures the mag¬ 
netic field produced by electrical activity in the brain 
(Figure 4.20). Its spatial resolution at parts of the 
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TABLE 4.1 EEG frequencies and their associated functions 


Name and example 


Description 


Delta 

MW 


Delta is the slow wave characteristic of deep, unconscious sleep. It is less than 4 Hz, and similar EEG frequencies 
appear in epileptic seizures and loss of consciousness, as well as some comatose states. It is therefore thought to 
reflect the brain of an unconscious person. 

The Delta frequency tends to be the highest in amplitude and the slowest waves. Delta waves increase in 
relation to our decreasing awareness of the physical world. 


Theta 



Theta activity has a frequency of 3.5 to 7.5 Hz. 

Theta waves are thought to involve many neurons firing synchronously. Theta rhythms are observed during 
some sleep states, and in states of quiet focus, for example meditation. They are also manifested during some 
short term memory tasks, and during memory retrieval. 

Theta waves seem to communicate between the hippocampus and cortex in memory encoding and retrieval. 


Alpha 

MW 


Alpha waves are those between 7.5 and 13Hz that arise from synchronous and coherent (in phase) electrical 
activity of large groups of neurons in the human brain. They are also called Berger's waves in memory of the 
founder of EEG. 

Alpha waves are predominantly found to originate from the occipital lobe during periods of relaxation, with eyes 
closed but still awake. Conversely alpha waves are attenuated with open eyes as well as by drowsiness and sleep. 



Beta activity is'fast'irregular activity, at low voltage (12-25 Hz). 

Beta waves are usually associated with normal waking consciousness, often active, busy, or anxious thinking 
and active concentration. Rhythmic beta with a dominant set of frequencies may be associated with various 
pathologies and drug effects. 

Beta is usually seen on both sides of the brain in symmetrical distribution and is most evident frontally. It may be 
absent or reduced in areas of cortical damage. 


Gamma 


Gamma generally ranges between 26 and 70Hz, centered around 40Hz. 

Gamma waves are thought to signal active exchange of information between cortical and subcortical regions. It 
is seen during the conscious waking state and in REM dreams (Rapid Eye Movement sleep). Note that gamma 
and beta activity may overlap. 


cortical surface is now approaching a few millimeters, 
while its temporal resolution is in milliseconds (Figure 
4.21). 

Because of the physics of magnetism, MEG is highly 
sensitive to dendritic flow at right angles to the walls 
of the sulci (the cortical folds), but much less sensitive 
to the bottom. MEG has excellent temporal resolution 
and somewhat better spatial accuracy than EEG. 

Like any other method that measures brain activity, 
MEG results must be superimposed upon a structural 
image of the living brain. MEG uses a process called 
magnetic source imaging (MSI) to co-register the mag¬ 
netic sources of brain activity onto anatomical pictures 
provided by MRI. In this way, MSI combines the high 
spatial resolution of MRI with the high temporal reso¬ 
lution of MEG. MSI techniques are used before brain 
surgery, to pinpoint brain regions with vital functions 
that must be protected during surgery. 


MEG has the advantage of being entirely silent and 
non-invasive. As we will see, MRI is quite noisy, and of 
course depth electrodes require surgery. Thus, MEG is 
attractive for use with children and vulnerable people. 

2.5 Zapping the brain 

We have discussed techniques for non-invasive record¬ 
ing of brain signals, but what if you could evoke neural 
activity in a safe fashion? Such a method would be 
especially useful to test causal relationships between 
evoked neural activity and cognitive functions. 

Early work on direct electrical brain stimulation began 
with Wilder Penfield, a neurosurgeon at the Montreal 
Neurological Institute (Figures 4.22 and 4.23). Penfield 
and his colleagues treated patients with intractable epi¬ 
lepsy. In open brain surgery, patients can remain awake 
and responsive, since only local anesthetic is needed at 
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FIGURE 4.17 Regular waveforms from different brain regions. Three waveforms with important cognitive func¬ 
tions are alpha, classically associated with relaxed and alert states, and often found over the occipital cortex. Frontal 
theta is now believed to reflect interactions between the hippocampus and cortex. Finally, gamma activity is often 
thought to reflect transient large-scale assemblies of neurons associated with conscious percepts and other cognitive 
processes. 'Spectral power' is the area under the curve of the frequency graph, and reflects the physical energy in each 
frequency band. Source : Ward, 2002. 


the site. There are no pain receptors in the brain itself, 
so that the cortex brain can be stimulated and operated 
upon without general anesthesia. This is a vital part of 
the operation, because surgeons need to map out brain 
regions that are needed for normal functions. Epileptic 
foci, which trigger major seizures, can then be cut out 
with the least side effects. 

2.5.1 A safe way of interfering with brain function: 
transcranial magnetic stimulation (TMS) 

It is now possible to stimulate brain lesions in healthy 
subjects. Without cutting a person's brain, we can alter 
the brain's level of activity very locally. Brief mag¬ 
netic pulses over the scalp either inhibit or excite a 
small region of cortex. For example, if you stimulate 
the hand area of the motor cortex, the subject's hand 
will suddenly move and twist. Applying an inhibitory 
pulse over the same area will cause subjects to have 


difficulty moving their hands. This is called tran¬ 
scranial magnetic stimulation (TMS) or, as one leading 
researcher has called it, 'zapping the brain' (Cowey 
and Walsh, 2001). TMS (Figure 4.26) appears to be gen¬ 
erally safe. By applying TMS, we can now test causal 
hypotheses about the contribution of specific brain 
regions to complex cognitive processes. Since the TMS 
works at the millisecond scale, it is also possible to 
study how rapid waves of processing develop. 

Overgaard et al. (2004) used TMS to explore visual 
consciousness. Subjects were given simple geometric 
shapes, like triangles, squares, and circles, in three dif¬ 
ferent colors (red, green, and blue). The stimuli were 
presented in one of three locations on the screen. All 
the shapes were shown long enough to be seen by all 
subjects. The researchers then applied a TMS inhibi¬ 
tory pulse to the mid-temporal region in both hemi¬ 
spheres, roughly just in front of each ear. If the pulse 
was applied about 120 milliseconds after the stimulus 
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FIGURE 4.18 Evoked potentials are often shown by topographical maps of averaged voltages across the scalp. Redder 
colors are higher voltages. In this case, explicit remembering of conscious memories evokes much higher ERPs than the "feel¬ 
ing of knowing' for the same memories. Brain activity is not as well localized with ERPs as it would be with PET or fMRI. 
Source: Diizel et ah, 1997. 
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FIGURE 4.19 Evoked potentials to musical meaning. Regular musical sequences are perceived as more pleasant, and show 
a larger evoked potential (see the arrow in the graph, (c)). Irregular musical sequences evoked a very different brain response. 
Source : Koelsch and Siebel, 2005. 


was shown, there was a dramatic drop in subjects' 
awareness of the stimuli. They no longer had a clear 
perception of the figures, but either a vague impres¬ 
sion that something had been shown, or none at all. 


Interestingly, when subjects were asked to guess 
the stimuli, they all showed normal performance. Yet 
they did not report being conscious of the stimuli. The 
stimuli must still have been detected (unconsciously) to 
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FIGURE 4*20 MEG is silent and non-invasive. MEG is easy for young children to toler¬ 
ate, as long as they can stay relatively still. The pictures at the bottom of the figure show the 
vector fields of the MEG over the head of the subject. Source : 4D Neuroimaging, San Diego. 
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FIGURE 4.21 An example of a wavelet analysis of MEG data in response to speech and non¬ 
speech sounds. The non-speech sounds consisted of steady-state tones, tonal contours (TC) which 
contained dynamic changes; speech sounds consisted of steady-state vowels and consonant vowel 
(CV) syllables which contained dynamic changes. Using a wavelet analysis, with time on the x- 
axis and frequency on the y-axis, and color denoting the level of activity, the authors investigated 
neural activity by stimulus contrasts and task (AUD, the subjects decided if two sounds were 
exactly the same, CAT, the subjects decided if two sounds were in the same category). Results: 
high levels of activity in the alpha (8-13 Hz) band, shown in red on the wavelet plots. 
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FIGURE 4.22 Wilder Penfield. Penfield and colleagues devised 
open-brain neurosurgery for untreatable epilepsy in the 1950s. 
Source : Adelman and Smith, 2004. 



FIGURE 4.23 Penfield's map of brain regions where electri¬ 
cal stimulation interferes with language. Penfield and Roberts dis¬ 
covered that electrical stimulation in the indicated regions interfere 
with language production or perception. Notice how closely these 
regions correspond to the classical Broca's and Wernicke's patients 
studied a century before. Source : Adelman and Smith, 2004. 


FRONTIERS OF COGNITIVE NEUROSCIENCE 


A conversation with Michael Rugg: Neuroimaging 
correlates of cognition 



FIGURE 4.24 Michael D. Rugg, PhD, Department of 
Neurobiology and Behavior and Department of Cognitive 

The emergent field of cognitive neuroscience provides 
exciting new and ever-changing ways to investigate the 
neural correlates of human thought and cognition. An 
influential leader in this field is Dr. Michael Rugg, Professor 
in the Department of Neurobiology and Behavior and 
Director of the Center for the Neurobiology of Learning 
and Memory at the University of California, Irvine. We sat 
down with Dr. Rugg and asked him how his approach to 


investigating the neural correlates of human memory have 
shifted over the past decade, and how he thinks the field 
will continue to shift in the next. Here are some highlights 
of that conversation. (CBC: editors Bernard J. Baars and 
Nicole M. Gage; MR: Michael Rugg). 

CBC: Let's begin with a little history: with respect to your 
own body of work, how has your approach changed 
in the past decade as a cognitive neuroscientist? 

MR: For me, I think it coincides with the availability of 
brain imaging methods that allow one to directly iden¬ 
tify the neural populations and the neural circuits that 
are engaged during different cognitive tasks. When 
I first started in what we now call cognitive neuro¬ 
science, it was primarily composed of people who 
wanted to use brain measures to dissociate cognitive 
processes. Pretty well the only method available was 
EEG-based measures, and so at that time there was 
a lot of emphasis on simply using neural measures 
to dissociate different kinds of cognitive processes. 
The logic is: if you have two experimental conditions 
that give rise to very different patterns of brain activ¬ 
ity that clearly don't come from the same 'bits of the 
brain', then that's reasonable evidence, given certain 
assumptions, that you may have dissociated two dif¬ 
ferent cognitive processes. 
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FIGURE 4.25 An example of the experimental 
design used in Dr. Rugg's fMRI memory experiments. In 
this experiment, they were investigating the role of atten¬ 
tion for color and location during a memory task. Top: 
Experimental design. Left: The study materials, showing 
the use of color and location cues during the Study Phase 
when subjects were scanned. Right : The study materi¬ 
als used during the Test Phase, where color and location 
cues were removed. Middle: Feature-specific subsequent 
memory effects. Left: Memory effects selective for color 
(displayed in yellow) are shown on a standardized brain. 
A subset of these color-selective memory effects was 
additionally modulated by attention to color (overlaid in 
green). Right: Memory effects selective for location (yel¬ 
low) also show brain areas that are modulated by attention 
to location (green). Bottom: Medial temporal lobe encoding 
attended contextual information. Left: A right hippocampal 
region shows memory effects for contextual information 
that is selectively attended (regardless of which feature is 
attended) and does not show significant effects for infor¬ 
mation that is not attended. Right: Graphs show estimates 
of the effects for attended vs. unattended trials, then bro¬ 
ken down by color and location. Source: Uncapher and 
Rugg, 2009. 
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And then with the shift to first PET and then 
fMRI, the same logic could be applied but now you 
have much more confidence in where in the brain 
these differential patterns of activation are occur¬ 
ring. You start to think much more directly about 
what different parts of the brain might be doing 
to support these different hypothetical cognitive 
processes: instead of using brain data as (more or 
less) a substitute for - or a complement to - reac¬ 
tion times or error rate, or any other behavioral 
measure, the data actually become a source of infor¬ 
mation in its own right. Now you may ask: why is 
it that certain kinds of retrieval operations engage 
different parts of the parietal cortex relative to 
other kinds of retrieval operations? What is it that 
parietal cortex might be doing that means that it is 
more interested in only some of these operations, 
and what does that tell us theoretically about the 
neural networks that support different kinds of 
cognitive processes? 

CBC: Investigating complex neural systems such as mem¬ 
ory presents a challenge for designing experiments 
that allow you to tease apart subtle but important 
aspects of a complex system. What is your approach 
to this? 

MR: I think one of the other things that has become clear 
is that you can't study memory - or the neural cor¬ 
relates of memory - without some reasonably tight, 
well-articulated preexperimental theory about how 
you think memory is organized and how you think 
your task is tapping into different kinds of memory 
processes. This is no different from any other area 
of cognition, but it is particularly painfully obvious 
in memory that you can't simply stick someone in 
the scanner and say T'm going to study memory.' 
You need to have a reasonably well-articulated cog¬ 
nitive or functional theory of memory which guides 
your experimentation and your interpretation. One 
of the big areas of controversy at the moment in the 
cognitive neuroscience of memory is really about 
this: it's not about the data, at all, the data are rea¬ 
sonably agreed-upon. It's about the interpretation of 


the data and that comes from people holding differ¬ 
ent preexperimental premises about how memory 
is organized, how it should be measured, and what 
the criteria are for saying that you have separated 
out differing memory processes from a behavioral 
point of view. Differing labs may have similar sets 
of data to other labs but their interpretation of 
those data may be starkly different: the different 
interpretations, then, depend not on the neuro¬ 
science, but on the experimental psychology. 

CBC: Let's look forward now, what do you see as 
upcoming changes in the approach to investigat¬ 
ing the cognitive neuroscience of human memory? 

MR: I think it's a question of getting more sophisticated 
in terms of the behavioral procedures and also think¬ 
ing much more deeply about what the theoretical 
implications are of differing patterns of findings and 
trying to see them from more than one theoretical 
perspective. The other thing that is clear - and this 
comes to a methodological point - it's probably no 
longer helpful to try to characterize memory-related 
activity in broad terms like 'hippocampus' vs. 
'non-hippocampus', 'entorhinal cortex' vs. 'para¬ 
hippocampal cortex', and so on. The reason being 
that these are highly differentiated structures and 
it's probably a gross oversimplification to think that 
just because two tasks both engage the hippocam¬ 
pus, for example, they must, in some sense, both be 
sharing some low-level cognitive operation. We now 
have the ability with MRI to do high resolution func¬ 
tional neuroimaging of the hippocampus at a level 
that allows us to identify patterns of activity in hip¬ 
pocampal subfields. There are people, for example, 
now claiming to see dissociable patterns of activ¬ 
ity in, say, dentate gyrus and CA1 vs. CA3. So how 
can you say, then, that those two tasks both engage 
the hippocampus and thus they are doing the same 
thing? In other words, the level of resolution, if you 
like, with which one can dissociate the neural cor¬ 
relates of cognition is getting finer and finer. That is 
going to change the way people conceptualize 'neu¬ 
ral dissociations' and 'cognitive dissociations.' 


permit accurate detection. Yet no subjective experience 
of the words was reported. 

Now let's turn to methods for investigating the 
brain that use spatial mapping techniques to explore 
both brain anatomy and function. 


3*0 FUNCTIONAL NEUROIMAGING: 

A BOLD NEW WORLD 

EEG and MEG measure brain activity fairly directly. 
Other neuroimaging measures use indirect measures. 
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Transcranial magnetic stimulation (TMS) 



Projection of cortical site onto head surface 

O Mesh nodes 
© Closest nodes 
^ Target projected node 


FIGURE 4.26 Magnetic brain stimulation is not surgically invasive. Brain stimulation can be applied without sur¬ 
gery. In this example, it is being applied over Broca's and Wernicke's areas in the left hemisphere of the subject. TMS 
appears to be safe at mild levels of intensity and frequency. It allows causal hypotheses to be tested in brain experiments, 
a major methodological advance. Source : Andoh et al., 2006. 



such as blood flow or regional oxygen level. Currently, 
the most popular method is fMRI (functional magnetic 
resonance imaging) (Figure 4.27) and especially the 
kind that measures the oxygen level of the local blood 
circulation (called BOLD, for blood-oxygen level 
dependent activity, see Figures 4.28 and 4.29). 

When neurons fire they consume oxygen and 
glucose and secrete metabolic waste products. An 
active brain region consumes its local blood oxygen 


supply, and as oxygen is consumed we can see as a 
small drop in the BOLD signal. In a fraction of a sec¬ 
ond the loss of regional oxygen triggers a new influx 
of oxygen-rich blood to that region. Here, we see a 
recovery of the signal. However, as the compensa¬ 
tory mechanism overshoots, flooding more oxygen¬ 
ated blood into the area than is needed, we also see 
that the signal rises high above the baseline. Finally, 
as unused oxygen-rich blood flushes out of the region. 
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FIGURE 4.27 How the MRI equipment looks to the sub¬ 
ject. Most £MRIs are taken with the subject lying down. Today's 
MRIs are still very noisy, as the electromagnetic coil is switched 
on and off. Small visual displays may be used to present stimuli, 
or headphones for auditory stimuli. Because the machine generates 
high-intensity magnetic fields, no metal objects like pens or even 
paperclips can be taken into the examination room. Source: Sharma 
and Sharma, 2004. 


we can see a drop in the BOLD signal, back to the 
baseline. 

Thus, as the oxygen content of blood produces 
changes in the BOLD signal, we can measure neural 
activation indirectly. The BOLD signal comes about 
six seconds after the onset of neuronal firing. The 
relationship between neural activation and the BOLD 
fMRI signal is shown in Figures 4.28 and 4.29. 

Positron emission tomography (PET) was developed 
much earlier than MRI or fMRI, and provides a meas¬ 
ure of metabolic brain activity (Figure 4.30). PET is used 
less often for research today, because it is very expen¬ 
sive, requiring a cyclotron. It also requires subjects to 
be injected with a radioactive tracer. For non-medical 
investigations, MRI and fMRI have largely taken over 
the research field (Figure 4.31). However, PET is still 
important, because different tracers can be linked to 
different molecules. The distribution of neurochemicals 
in receptors can therefore be determined. 

Today, it is not possible to have high spatial and 
high temporal resolution at the same time using the 
same recording device. There is a tradeoff. Methods 
like fMRI and PET tell us where in the brain something 
is happening. EEG, MEG and single cell recordings 
show millisecond changes in the brain. In any study. 


Blood Oxygenation Level Dependent (BOLD) effect 




FIGURE 4.28 The basis of functional MRI (fMRI). The most 
popular brain imaging method today is probably fMRI, which is 
less expensive than PET, and provides a good localization of brain 
activity. fMRI is an indirect measure of neuronal regional activity, 
as shown here. Neuronal activation increases the oxygen demand of 
neurons and related cells, leading to additional blood flow carrying 
oxygen molecules to the region. This can be measured using BOLD - 
Blood Oxygen Level Dependent activity. Since neurons start firing 
several seconds before additional BOLD activity starts, there is a 
built-in lag time in fMRI. Source: Dogil et al., 2002. 


it is important to ask why the authors chose a particu¬ 
lar method, and what they observed - or might have 
missed - by the choices they made. In the best cases we 
can see different methods showing convergent results. 

3.1 Regions of interest 

Finding your way around the brain is not easy. An even 
harder task is to figure out which areas play which 
roles in major cognitive processes such as language, 
attention and vision. One way is to define regions of 
interest (ROIs) ahead of the research study, and to 
make predictions about expected activity in ROIs. 

3.1.1 C o-registration of functional activity on a 
structural scan 

The first step is to define the living brain anatomically, 
to make out different areas, connections and layers of 
organization. Structural MRI gives us a tool to map out 
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FIGURE 4*29 A typical BOLD response to a neural population. The top line shows a burst of activity in a 
population of neurons somewhere in the brain. In a few seconds, the active brain region has used up its imme¬ 
diate supply of nutrients - oxygen and glucose. The BOLD curve therefore dips to reflect the loss of oxygen. 
Next, an upward sweep of the BOLD curve reflects a wave of new, blood-carried nutrients to the active region to 
keep it going. This wave of oxygen is used up again by energy-thirsty neurons, and the curve dips down again. 
Eventually, the nutrient supply comes to equilibrium with the neural demand, and the curve goes back to baseline. 
The BOLD response is called 'hemodynamic' because it reflects fast and precise changes in the local blood supply. 
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FIGURE 430 A classical PET finding: visual versus auditory brain activity. Early PET scans showing 
different speaking, seeing, hearing, and internally generating words (Posner and Raichle, 1994). Notice that 
visual, auditory, motoric, and speech production regions appear to be activated. However, the surrounding 
brain outline (white lines) is only approximate. In more recent brain images, the functional activity would 
be superimposed upon a structural MRI of the same subject's brain. Source : Posner and Raichle, 1994. 
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fMRI SHOWS THE ACTIVITY OF THE BRAIN 


Left Hand - Touch 



2 minutes 24 seconds 2 minutes 24 seconds 
TIME 


FIGURE 4.31 How the BOLD signal cycles on and off. The basic physics of BOLD 
requires a high magnetic field to be turned on and off frequently to detect the radio fre¬ 
quency changes due to spin changes in oxygen atoms. Because cognitive tasks occur over 
seconds and fractions of seconds, a common method is to alternate Task and Rest condi¬ 
tions every half minute or so (in this case, 40 seconds per phase). This allows for excellent 
within-subject control. On the left, the functional signal (i.e. BOLD) is superimposed on a 
structural brain scan (using MRI). All brain images superimpose the task-related activity 
upon brain structure, and average out background activity. Source : Robinson, 2004. 



FIGURE 4.32 The BOLD fMRI signal for pain on the left side 
of the body. By stimulating the left hand to produce mild pain, a 
BOLD signal emerges in the right-side body map (somatosensory 
cortex, not shown), and also in these areas along the midline of the 
right hemisphere. The pain pathways are well known, and mild 
pain perception is an easy way to test the fidelity of a brain imaging 
technique. Source: Valet et al., 2004. 

brain structure, including the axonal (white matter) con¬ 
nections between brain regions. MRI shows structure but 
not function. 

On the right side of Figure 4.32, we see an image with 
two yellow 'hot spots', reflecting increased fMRI activ¬ 
ity. The color is arbitrarily chosen to indicate the degree 


of activity. In order to pin down the location of the yel¬ 
low hot spots, we need to superimpose the functional 
image on the structural MRI, which has a better spatial 
resolution. In a process called co-registration , the func¬ 
tional and structural images are aligned to each other. 
Co-registration ensures that the two images end up in 
the same space using the same metric. With higher spa¬ 
tial resolution we can ask questions that are anatomi¬ 
cally specific. 

Another approach is to mark regions of interest 
(ROI) on the structural image alone, to constrain the 
statistical analysis. 

Newer MRI machines (Figure 4.27) with higher mag¬ 
netic field strength now make it possible to look at the 
cellular organization of the living brain, and to compare 
brains between groups of people (e.g. people with schiz¬ 
ophrenia and healthy subjects). Different layers of cortex 
have either local or distant connectivity, so that layer 
information is useful to find out how cortical regions 
interact. 

Because the brain is remarkably active at all times, it 
is still a challenge to isolate fleeting, task-related activ¬ 
ity. One common method is subtraction between fMRI 
conditions: for example, the brain's response to the task 
of interpreting Arabic numerals (1, 2, 3, . . .) is com¬ 
pared to its response to the same numbers when they 
are spelled out in printed words (one, two, three, . . .) 
(Figure 4.33). Subtraction is used because it tends to 
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PET and fMRI SUBTRACTION METHOD 
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FIGURE 4.33 The subtraction 
method for PET and fMRI. The brain 
has constant dynamic background 
activity. To remove background activ¬ 
ity, the BOLD or PET signal for an 
experimental task is subtracted, point 
by point, from a closely-matched con¬ 
trol task. Individual scans of the differ¬ 
ences are then averaged, and produce 
the group average. Source: Posner and 
Raichle, 1997. 


remove most of the 'irrelevant' brain activity, which 
would otherwise drown out the signal of interest. It is 
much like comparing a choppy sea before and after a 
whale swims by If you want to see the waves gener¬ 
ated by the whale alone, you might subtract a record 
of the waves alone. It is the difference between the two 
conditions that matters (see Figure 4.33). 

Subtracting conditions can have unwanted conse¬ 
quences. There might be important things going on 
in both conditions. In addition, the variance of experi¬ 
mental and comparison conditions might be differ¬ 
ent, there might be interactions between independent 
variables, and so on. Another approach therefore is 
parametric variation, in which the variance for each 
main variable and their interactions, can be separated 
statistically. For example, if you study working mem¬ 
ory (WM), instead of subtracting brain activity dur¬ 
ing working memory from activity without it, one can 
study how gradually increasing the WM load leads to 
changes in neural activation. Since statistical testing 


must be done for every point of interest in the scan, 
over every unit of time, this is a large data set. 

3*2 The resting brain is not silent: intrinsic 
brain processes 

As neuroimagers began to study different cognitive func¬ 
tions, the main approach was to use a contrastive, sub¬ 
tractive approach. Here, the neural activation during a 
given cognitive function, such as speech production, was 
compared to a period where subjects were instructed to 
relax and 'do nothing'. Such active-versus-rest compari¬ 
sons showed powerful main effects. Hundreds of com¬ 
parison studies have been conducted to map the brain 
regions for cognitive functions. 

Yet there is a hidden assumption in these studies. 
If you are asked just to lie still and 'rest' what will 
you do? Will your mind be a blank screen? A great 
deal of evidence shows that people just go back to 
their everyday thoughts, images, and feelings. You 
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(b) 


Face matching > control 


■ Location matching 
> control 


FIGURE 4*34 A typical visual experiment using fMRI. Note that the visual faces are closely matched with vis¬ 
ually similar stimuli. The face stimuli are compared to nonface objects in the same spatial orientation. Subjects need 
to pay attention, so a matching task is used, in which they press a button when the matching stimulus appears. 
The results show higher BOLD signals for faces in red areas, higher for location in the green areas, and higher 
face + location in the yellow-shaded parts of the brain. Source: Squire et al., 2003. 



might be thinking about something that happened 
the day before; or that you need to do later, or even 
daydream a bit. You are likely to have inner speech, 
visual imagery, episodic memories coming to mind 
and so forth. For the brain, that is not 'rest'. Instead, 
the experimental comparison is really between two 
active states of the brain. One is driven by experimen¬ 
tal task demands, while the other reflects our own 
thoughts, hopes, feelings, images, inner speech, and 
the like. In some ways, spontaneous activity may tell 
us more about the natural conditions of human cogni¬ 
tive activity than specific experimental tasks. Both are 
obviously important. 

An important literature has now sprung up to 
study the cognitive 'rest' condition, i.e. a condition 


under which subjects are not asked to do anything in 
particular (Figure 4.35). 

The use of MRI to produce both precise anatomi¬ 
cal images and to provide functional maps of brain 
areas has literally revolutionized the field of cognitive 
neuroscience. New and better ways to use fMRI are 
presented in the following section. 

33 Empirically defining cognitive functions: 
the creative key 

The best science is done by combining imaging tech¬ 
niques with genuine creativity. A lot of creativity goes 
into the selection of functional variables. What is the 
best way to understand vision? Selective attention 
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FIGURE 4.35 The brain is an active place. 
This fMRI shows spontaneous brain activity in both 
the left hemisphere (left) and the right hemisphere 
(midline view, right). The bottom shows these activ¬ 
ities over 300 seconds. While such Task-unrelated' 
activity is different from typical experimental tasks, 
we know that humans are constantly thinking, 
imagining, feeling, anticipating, remembering, and 
talking to themselves, even without being given a 
specific task to do. Source: Fox et al., 2005. 


and conscious cognition? A great deal of ingenuity has 
been devoted to those questions. 

Here are a few examples. For example, taxi driv¬ 
ers are well known for their ability to know their way 
around a city. They know not only how to get from A 
to B, but also the most efficient way to get there. Such 
ability to navigate through a complex road system 
depends on our spatial ability. Studies have shown that 
the hippocampus, a part of the medial temporal lobe, 
plays an important part in the navigation and memory 
of places and routes. Rats with lesions to the hippoc¬ 
ampus have been known for decades to perform dis¬ 
astrously on spatial tests. Birds and other animals that 
bury or hide their food at multiple places have larger 
hippocampi than non-storing animals. Therefore, one 
question that arises when we think about taxi driv¬ 
ers is, are the brain regions responsible for spatial 
navigation more developed in taxi drivers than other 
people? Indeed, it has been found that part of the hip¬ 
pocampi (posterior) of taxi drivers was larger than 
the same region in a group of people with a different 
background (see Figure 4.36). OK, you might ques¬ 
tion, but what if people with large hippocampi choose 
to be taxi drivers, and not vice versa? Here, the study 
showed that the size of the hippocampus depended 
on how long people had been working as taxi drivers. 
In other words, the longer you work as a taxi driver 
(and use your spatial navigation ability) the bigger 
your relevant part of the hippocampus will become. 

Notice how imaginative the taxi driver study was. 
It is usually easier to randomly select human subjects 
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Anterior Body Posterior 

FIGURE 4*36 Hippocampal size in London taxi drivers. London 
taxi drivers showed substantial differences in the size of a spatial 
region of the brain, the posterior hippocampus. Source : Maguire et ah, 
2000 . 
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(usually undergraduate students!) to stand for the entire 
human population. But the fact is that there are differ¬ 
ences of age, particular abilities and talents, and other 
cognitive capacities between 'average' subjects. London 
taxi drivers are highly experienced experts (they are 
required to pass tests on the geography of the city), and 
they can be compared to a plausible control group. One 
important implication is that the sheer size of brain struc¬ 
tures may change with specific experiences (Maguire 
et ah, 2000). That claim has now been supported 
for other brain regions as well. The taxi driver study is 
therefore an excellent example of creative selection of 
comparison conditions, leading to new insights. 


4.0 NEW WAYS TO MEASURE BRAIN 
CONNECTIVITY: DIFFUSION TENSOR 
IMAGING 

White matter fiber tracts are the vast internal highway 
system of the cortex. We can visualize these fiber tracts 
using an MRI method called diffusion tensor imag¬ 
ing (DTI). DTI uses water flow in the white matter to 
measure the relative direction of white matter tracts. 
This information allows us to investigate connectiv¬ 
ity patterns across the two hemispheres and along the 
four lobes of the brain (see appendix for more details 
on how this is measured). DTI helps us understand 
brain connectivity patterns in the healthy brain as well 
as investigate these patterns in individuals with brain 
diseases that affect white matter, such as multiple scle¬ 
rosis (MS). 

By far the largest fiber bundle in the brain is the cor¬ 
pus callosum (see Chapter 5), which connects the two 
hemispheres, but there are many other fiber bundles 
or tracts that connect regions within the hemispheres. 
Look at the figure at the beginning of this chapter - it 
shows a map of colored fibers that spread up from the 
corpus callosum through cortex. The scientists who 
prepared this figure color-coded the fibers in the dif¬ 
ferent brain regions, with green fibers in the frontal 
lobe, moving to yellow in the occipital lobe. Another 
view of the vast array of white matter tracts is shown 
in Figure 4.37, with a midsagittal (center-line) view 
of the fiber tracts that extend upward from the spi¬ 
nal cord to cortex and tracts that extend downward 
from cortex to the spinal cord. These fiber tracts make 
up the vertical Traffic arteries' that flow to and from 
the cortex and that provide the connective pathways 
throughout the central nervous system. What is the 
relationship between fiber tracts and cortical anatomy? 



FIGURE 4.37 A fountain-like image of the white fiber tracts. 
This beautiful spray of neuronal tracts results from a magnetic imag¬ 
ing technique called diffusion tractography, which allows us to view 
the white (myelinated) fiber tracts. These are the vertical traffic arter¬ 
ies that flow from the cortex (especially frontal ones, in blue) down 
into the spinal cord; and upward from the spinal cord to the rear half 
of cortex (in yellow and green). Most of the volume of cortex is taken 
up by these massive fiber tracts. Source : Maria Lazar, with permission. 

A recent study investigated the correspondence 
between white matter and gray matter and results are 
presented in Figure 4.38. Constructed maps of periph¬ 
eral white matter tracts are presented at the top of the 
figure and their corresponding cortical gray matter 
regions are shown below. Note that Figure 4.38 shows 
the same information across three views of the brain: 
from the front or anterior perspective (A and D), from 
the same or lateral perspective (B and E), and from the 
back or posterior perspective (C and F). 

5.0 CONSCIOUS VERSUS 
UNCONSCIOUS BRAIN EVENTS 

Conscious cognition is a new direction in cognitive neu¬ 
roscience. It involves trying to understand the differ¬ 
ence between conscious and unconscious brain events. 
In the last decade or two, scientists have made use of 
many experimental paradigms to compare the two. 
One major example is binocular rivalry, presenting a 
stream of two different visual stimuli to the two eyes, so 
that only one stream of input is conscious at any given 
time. Behaviorally, humans can only report one stream 
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FIGURE 4.38 Diffusion tractography brings out the corpus 
callosum and other large fiber tracts. Notice that we are looking 
at a cutaway image of the brain, in which the right hemisphere is 
intact. We can see the green-marked fibers projecting upward in 
front, and yellow-marked ones projecting toward the rear. Different 
artificial colors are assigned by computer to different directions of 
travel of the fiber highways. The c-shaped or banana-shaped struc¬ 
tures in section (b) are the corpus callosum (the "calloused body'), 
which looks white to the naked eye. The corpus callosum contains 
about 100 million fibers, running sideways from one hemisphere to 
the other. Millions of cells in the left hemisphere connect to a corre¬ 
sponding point in the right hemisphere. Source : Huang et al, 2005. 


(see Chapter 6). Binocular rivalry works in macaque 
monkeys much the way it does in humans (Logothetis, 
1998), and the wave of current interest in visual con¬ 
sciousness may be dated to a classic series of binocular 
rivalry studies in macaques, using single neuron record¬ 
ing at different levels of the visual hierarchy (Sheinberg 
and Logothetis, 1997); see also the discussion of binocu¬ 
lar vivalry in Chapter 6. 




6.0 CORRELATION AND CAUSATION 

We typically combine brain and behavioral observa¬ 
tions. We can present visual images on a screen and 
have the subject read aloud or meditate. Thus, we typi¬ 
cally observe a correlation between behavior and brain 
activity. In methods with high spatial resolution, such as 
fMRI, different tasks show local increases or decreases in 
the BOLD signal, indicating that the brain works more 
in some regions than others (see Figure 4.31). 

We can take the example of the 'Counting Stroop 
Task', in which subjects are asked simply to count the 
number of words shown on a computer screen. On 
some occasions a word like 'dog' is shown three times. 
The subject should say 'three', which is not difficult. 
However, we can introduce a conflict between the words 
shown and the number of words. If we show the word 
'one' four times, there is an automatic tendency for expert 


FIGURE 4*39 Causal relationships between brain activities dur¬ 
ing the Stroop task. Brain activity during simple counting with no 
interference is shown at the top (a), and the activation during counting 
with interference is shown at the bottom (b). Each circle represents a 
region of the brain. As can be seen, interference (b) leads to the engage¬ 
ment of a more widespread network than the control condition (a). 
This change in causal coupling between brain areas is seen in spite of 
the fact that many of the same areas are active in both conditions. Note 
also that some of the connections are lost between the control and the 
interference condition. (Adapted from Zheng and Rajapakse, 2006.) 

readers (like college students) to say the word 'one'. But 
the correct answer is 'four'. This is very much like the 
Color-naming Stroop Task (see Chapter 2). Subjects take 
longer to answer correctly, since they must inhibit their 
automatic tendency just to read the words on the screen. 

Zheng and Rajapakse (2006) reported the BOLD activ¬ 
ity during the two versions of word counting (Figure 4.39). 
While many brain regions show activation during both 
conditions, frontal parts of the brain were more active 
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during the conflict condition. This kind of result has 
now been found many times for conflictual tasks. One of 
the major roles of prefrontal cortex is to resolve conflict¬ 
ing tendencies, like the automatic tendency just to read 
words, against the tendency to follow the experimental 
instructions. Thus, we have a correlation between (a) 
frontal activation, (b) longer reaction times, (c) a sense 
of subjective effort, (d) a greater number of errors in the 
conflict condition. These are significant results, since 
there are many real-life conditions where conflicting ten¬ 
dencies need to be regulated. 

However, so far we have no way to test causal 
hypotheses. For example, we know that the task requires 
visual word recognition, response preparation, choosing 
between two possible answers, perhaps detecting con¬ 
flict, stopping the wrong answer from being said, select¬ 
ing the right answer instead, and so on. An approach 
called dynamic causal modeling (DCM) is one to analyze 
for causal relationships. Zheng and Rajapakse (2006) per¬ 
formed DCM on the brain activation they found in both 
word counting tasks. As you can see in Figure 4.39, DCM 
suggested that each task had a different activation pat¬ 
tern. Although many of the same regions of the brain are 
active during both tasks, their relative connectivity and 
contribution was altered. Interestingly, the analysis also 
showed that the interference condition recruits wider 
activity than the control condition. This is another com¬ 
mon finding for mentally effortful conditions (Duncan 
and Owen, 2000). 

6. 1 Why we need multiple tests of brain 
function 

According to some media headlines, brain scientists 
recently discovered the parts of the brain used for 
deception and lying. This kind of headline comes from 
studies like the one shown in Figure 4.40. It shows 
fMRI differences between brain regions that had 
greater BOLD activity when people were telling the 
truth (green) and cortical areas when they were made 
to tell a lie. Such experiments often use playing cards, 
and ask a group of subjects to Tie' by reporting a differ¬ 
ent card than the one they actually see. 

One major purpose of cognitive neuroscience is to 
identify function with structure, i.e. to see whether 
specific brain locations do specific things for us. In 
that process, however, we must be very clear about 
the kinds of inferences that we can make from the evi¬ 
dence. The popular media may not be quite as careful 
as scientists need to be. Do you believe that the green 
areas in Figure 4.40 are really the Truth telling' areas 
of the brain? 



FIGURE 4.40 Are these the truthful and the deceptive areas of 
the cortex? fMRI differences between brain regions that had greater 
BOLD activity when people were telling the truth (green) and corti¬ 
cal areas when they were made to tell a lie (red). Is this the truth¬ 
telling cortex (green) and the lying cortex (red)? Why or why not? 
Source : Davatzikos et al., 2005. 
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Broca's area Wernicke's area 



FIGURE 4*41 Language area lesions. Lesions to either Broca's or Wernicke's areas produce very different 
effects on language. Notice that the right hemisphere has no Broca's and Wernicke's areas, traditionally, even 
though it is involved in the comprehension of speech and language. Source: Standring et al., 2005. 


6.2 Brain damage and causal inferences 

Brain injuries can provide evidence areas that are nec¬ 
essary for certain cognitive functions. Historically, 
Paul Broca discovered patients who were unable to 
speak and also showed damage to the left frontal lobe. 
However, their ability to comprehend language was 
relatively spared. About the same time, Carl Wernicke 
made the discovery that damage to different regions 
of the left hemisphere was associated with the ability 
to understand language, while their ability to speak 
was intact. Today, we call this complementary pair of 
facts a double dissociation. 

Brain damage is correlational, since we cannot pro¬ 
duce it and test its results in humans. Nevertheless, 
after a great deal of debate and controversy, there 
is no serious doubt today that one important func¬ 
tion of Broca's area is the production of speech, and 
likewise, that one important function for Wernicke's 
region is speech comprehension (Figure 4.41). Lesions 
near Broca's area can lead to dysarthria , a condition 
where the control of mouth and tongue movement 
is disrupted, but language production is still intact. 
Thus, we have three lesions that lead to three differ¬ 
ent speech problems: one seems to be important for 
language comprehension; another is vital for language 
production; and a third region is important for the 
motor commands to produce vocal expressions. 

Brain injuries in humans happen for a host of rea¬ 
sons, ranging from car accidents to strokes. Accidental 
lesions do not neatly follow the borders between brain 
regions. To test brain-mind hypotheses it is preferable 
to be much more precise about exactly where in the 
brain deficits occur. In order to do this, studies have 
been conducted in experimental animals, typically rats 
and monkeys. However, language is a species-specific 
function for humans, and we have no direct parallels 


in other species. (However, as mentioned above, there 
is now a way to produce safe interference in specific 
brain areas, using TMS.) 

Very precise lesions have been studied in monkeys 
and rats for many years, with significant results. For 
example, Buckley and Gaffan (2006) made precise 
lesions in different areas of the medial temporal lobe 
(MTL) in macaque monkeys. Very specific damage to 
the perirhinal cortex (meaning 'near the smell cortex') 
caused monkeys to make more errors on a visual object 
discrimination task. Lesions to surrounding areas did 
not produce this deficit. The harder the discrimination 
task became - the more alike the visual objects were - 
the more errors were made by the lesioned monkeys. Yet 
the monkeys performed normally in simple visual dis¬ 
criminations between different colors, shapes and ori¬ 
entations. This suggests that the perirhinal cortex may 
have a specific causal role in processing complex visual 
objects like faces. A variety of human studies now sup¬ 
port this finding. Animal studies have often played 
this pioneering role, allowing us to pick up leads that 
are later tested and often verified in humans. 

Comparing damaged and healthy brains is a pow¬ 
erful source of evidence for understanding basic neu¬ 
ral processes. 

7*0 SUMMARY 

Brain techniques measure single neurons to large corti¬ 
cal regions, brain structure, dynamic brain activity, and 
connectivity. The advent of brain imaging has trans¬ 
formed the study of human cognition. New and more 
refined methods are constantly being produced. One 
recent advance is to use multiple methods in the same 
study, so as to optimize the tradeoffs between electro¬ 
magnetic and indirect measures of brain activity. 
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8*0 CHAPTER REVIEW 

8* 1 Drawing exercises and study questions 

1 Label the differences between the brain scans in 
Figure 4.42, and describe the reasoning of the 
subtraction method for each image. 

2 Define the BOLD response. What does the 
abbreviation BOLD mean? 

3 What is the time lag between neural activity and 
a BOLD response? Between neural activity and 
an EEG response? 


4 What are the pros and cons of single-cell record¬ 
ing in the brain? 

5 What problem might arise when brain activity in 
a cognitive task is compared to a resting baseline? 

6 What does Figure 4.40 tell us about lying and 
the cortex? 

7 Judging by diffusion tractography, how much 
tissue is devoted to connections between corti¬ 
cal neurons (approximately)? 

8 Describe four common brain electrical rhythms. 
What functions are they often associated with 
(see Table 4.1)? 



FIGURE 4.42 A reproduction of Figure 4.33 shown without labels for use in the Drawing 
Exercises. 





The brain is contained within the cranium , and constitutes the upper , greatly expanded part of 
the central nervous system . 


Henry Gray (1918) 




Looking through the gray outer layer of the cortex, you can see a mass of white matter. At 
the center is a cluster of large nuclei, including the basal ganglia, the hippocampi, the 
amygdalae, and two egg-shaped structures at the very center, barely visible in this figure, 
the thalami. The thalami rest on the lower brainstem (dark and light blue). You can also see 
the pituitary gland in front (beige), and the cerebellum at the rear of the brain (pink). In this 
chapter we will take these structures apart and re-build them from the bottom up. 
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1.0 INTRODUCTION 

Our brains give us our biggest evolutionary edge. 
Other large mammals have bigger muscles and greater 
speed, but humans have an exceptionally big and flex¬ 
ible brain, specialized for excellent vision and hear¬ 
ing, language and social relationships, and for manual 
control and flexible executive control. Human brains 
make culture and technology possible. 

In this chapter, we look at the structure of the brain, 
while in the coming chapters we will cover its func¬ 
tions - how it is believed to work. It is important to 
understand that brain anatomy is not a static and 
settled field: new and important facts are constantly 
being discovered. On the microscopic and nanoscopic 
levels, whole new classes of neurons, synapses, con¬ 
nection patterns, and transmitter molecules have been 


found. While knowledge of the brain is constantly 
expanding, we will focus on the basics. 

Cognitive neuroscience inevitably focuses on the 
cortex, often considered to involve the 'highest level' 
of processing. The cortex is only the outer and vis¬ 
ible portion of an enormous brain, one that has devel¬ 
oped over hundreds of millions of years of evolution. 
The word 'cortex' means bark, since that was how it 
appeared to early anatomists. While the cortex is vital 
for cognitive functions, it interacts constantly with 
major 'satellite' organs, notably the thalamus, basal 
ganglia, cerebellum, hippocampus, and limbic regions, 
among others. The closest connections are between 
the cortex and thalamus, which is often called the tha- 
lamo-cortical system for that reason. In this core system 
of the brain, signal traffic can flow flexibly back and 
forth, like air traffic across the earth. 
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5. THE BRAIN 


The major lobes of cortex are comparable to the 
earth's continents, each with its population centers, nat¬ 
ural resources, and trade relations with other regions. 
While cortical regions are often specialized, they are 
also densely integrated with other regions, using web¬ 
like connections that spread throughout the cortex and 
its associated organs. This outer sheet is called the gray 
matter from the way it looks to the naked eye. It is the 
outer 'skin' of the white matter of cortex which appears 
to fill the large cortical hemispheres, like the flesh of a 
fruit. However, this is only appearance. In fact, the gray 
matter contains the cell bodies of tens of billions of 
neurons that send even larger numbers of axons in all 
directions, covered by supportive myelin cells that are 
filled with white lipid molecules. These white myelin 
sheaths of cortical neurons are so pervasive that they 


make the connections between cortical neurons look 
white to the naked eye. 

1.1 The nervous system 

The brain is part of the nervous system which per¬ 
vades the human body. The two main parts are the 
central nervous system (CNS), which includes the 
brain and the spinal cord, and the peripheral nerv¬ 
ous system (PNS), which contains the autonomic and 
peripheral sensory and motor system (Figure 5.1). 

Together the CNS and PNS provide a dynamic and 
massive communication system throughout all parts 
of the body, with a hub at the brain that is accessed 
through the spinal cord. We will focus in this chapter 
on one part of the CNS, the brain (Figure 5.2). 


Peripheral nervous Central nervous 



FIGURE 5.1 The central and peripheral nervous systems. 
Source : Standring, 2005. 
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FIGURE 5.2 Parts of the central nervous system include the 
spinal cord and the brain. Source: Standring, 2005. 
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In this chapter, we will focus on two sensory 
input systems within the brain: vision and hearing. 
Although there are other sensory input systems, such 
as olfaction (smell) and somatosensory (touch), vision 



FIGURE 5.3 The location of the brain in the head, showing a 
midsagittal view of the right hemisphere. 


and hearing have been most studied in the human 
brain. We will focus on two output systems, speech 
and hand-arm control, again because they have been 
the target of much study. Throughout this chapter on 
the brain, we will describe the anatomy of the brain 
and brain regions and we will also highlight the func¬ 
tion they serve. We will begin with discussing the 
many levels of analysis that we can take in describing 
the brain - from large-scale regions such as cerebral 
hemispheres and cortical lobes, to finer-scale classifi¬ 
cations, such as cortical layer topography. 


1.2 The geography of the brain 

Let's begin with the large-scale brain areas and work 
our way down to a finer analysis. First, to state the 
rather obvious, the brain is located in the human head, 
as depicted in Figure 5.3. 

We can look at the brain at different geographical 
levels - from continents to countries, states, and cities. 
Thus, we have several levels of detail. The first dis¬ 
tinct geographical regions are the two cerebral hemi¬ 
spheres, which are entirely separate, joined through a 
complex connective region called the corpus callosum. 
We will discuss the hemispheres in more detail later in 
the chapter: the question of why we have two separate 
hemispheres in the brain has long intrigued scientists 
and philosophers alike. 

Next, we have the cortical lobes (Figure 5.4): there 
are four lobes in each hemisphere. Beginning at the 



FIGURE 5.4 The four major lobes 
of the cortex are visible from a lateral 
view of the brain. Here we show a view 
of the left hemisphere with the frontal 
lobe (purple) at the anterior of the brain, 
the parietal lobe (orange) posterior to 
the frontal lobe at the superior aspect 
of the brain, the temporal lobe (blue) 
posterior to the frontal lobe and inferior 
to the parietal lobe, and the occipital 
lobe (yellow) posterior to both the pari¬ 
etal and temporal lobes. Just below the 
occipital lobe is the cerebellum (green), 
which is not part of the cortex but is 
visible from most aspects of the brain. 
Source : Squire et ah, 2003. 
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FIGURE 5.5 Some important landmarks of the brain in the left hemisphere from a lateral perspective (left panel) and a midsagittal 
perspective (right panel). Source : Standring, 2005. 



FIGURE 5.6 The Brodmann classification of regions in the left 
hemisphere, shown in a lateral view. Areas 41 and 52 are indicated 
by lines. Some areas, like the insula and auditory region, are tucked 
away behind the temporal lobe. 


front or anterior part of the brain (shown on the left 
side of Figure 5.4), we see the frontal lobe. Immediately 
behind the frontal lobe, at the top or superior part of 
the brain, we find the parietal lobe. Below, or inferior 
to, the parietal lobe and adjacent to the frontal lobe, 
we find the temporal lobe. At the back or posterior part 
of the brain, we find the occipital lobe. We will discuss 
the anatomical features and cognitive function of these 
lobes later in the chapter. 



FIGURE 5.7 The Brodmann classification of regions in the 
right hemisphere, shown in a midsagittal view. 


We can see the major lobes with the naked eye, 
along with their hills and valleys, the gyri and sulci. 
Some of these are so important that we will call them 
landmarks - we need to know them to understand 
everything else. In Figure 5.5, we show some of the 
major landmarks that brain scientists have long used 
to identify regions in the brain. These landmarks are 
widely used today when discussing the results of neu¬ 
roimaging studies. 

At a more microscopic level of description, we have 
the Brodmann areas, the numbered postal codes of the 
cortex. When the surface layers of cortex are carefully 
studied under a microscope, small regional differences 
can be seen in the appearance of cells in the layers and 
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their connections. Those subtle differences were first 
described by Korbinian Brodmann in 1909, and are 
therefore known as Brodmann areas, each with its own 
unique number (Figure 5.6 shows a lateral view of 
Brodmann areas in the left hemisphere, and Figure 5.7 
a medial (midsagittal) view of the Brodmann areas 
in the right hemisphere). About 100 Brodmann areas 
are now recognized, and it is therefore convenient to 
take this as a rough estimate of the number of special¬ 
ized regions of the cortex. The Brodmann areas cor¬ 
respond well to different specialized functions of the 
cortex, such as the visual and auditory areas, motor 
cortex, and areas involved in language and cognition. 
They are essentially the postal codes of the brain. They 
range in size from a few square inches - the primary 
visual cortex, for example, is about the size of a credit 
card - to the small patch of Brodmann area 5 at the 
top of the somatosensory cortex. 

Notice that in Figure 5.6, with the brain facing left, 
neighboring Brodmann areas are colored to show 
their major functions including vision, hearing, olfac¬ 
tion, motor control, Broca's area (speech output), 
and Wernicke's area (speech perception and compre¬ 
hension). This figure will be used as a reference map 
throughout this book. 

We can focus even more specifically by observ¬ 
ing hypercolumns, columns, and single neurons. 
At this fine level of resolution the current standard 
is the Talairach coordinates (Talairach and Tournoux, 
1988), which is used in functional brain imaging. The 
Talairach system can be compared to the map coordi¬ 
nates of the world, as shown on a GPS locator. They 
indicate the street addresses of the brain. 

It helps here to refer back to Figure 4.3, p. 99. The 
fine red lines show the axes of a three-dimensional 
coordinate system. On the upper left, we see the 
medial view of the right hemisphere, looking to the 
left (see the small head inset for orientation). In this 
image, the horizontal red line always runs between 
the pineal body (not visible), and the small cross- 
section of the anterior commissure - one of the tiny 
white fiber bridges that run between the two sides of 
the brain. The three-dimensional zero point (0, 0, 0) 
of the coordinate system is always at the front of these 
two points. This allows all three dimensions to be 
defined with good reliability. Notice the three stand¬ 
ard perspectives on the brain: the medial view 
(midsagittal), the horizontal or axial, and the coro¬ 
nal (crown-shaped) cross-slice. This software display 
allows any point in the brain to be specified pre¬ 
cisely, by entering numbers for the points in three 
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FIGURE 5.8 The six major layers of cortex in cross section. 
The figure shows three columns in Area 17, also called VI, the first 
visual projection area to the cortex. Source : Squire et al., 2003. 

dimensions. While human brains vary a great deal, 
much as human faces do, the Talairach system allows 
different brains to be mathematically 'squeezed into a 
shoebox', so that different brains can be compared in a 
single framework. 

Let's continue our description of the geography of 
the brain with a look at the fine structure of the cor¬ 
tex. The visible outer brain consists of a large thin 
sheet only six cellular layers thick (Figure 5.8), called 
the cortex (meaning 'bark', like the outside bark of 
a tree). This sheet is called the gray matter from the 
way it looks to the naked eye. Not all cortex has six 
layers; only the giant mammalian cortex does, and is 
therefore sometimes called 'neocortex' (That is, the 
new cortex, because it only emerged 200 million years 
ago!) Older regions of cortex are also found in rep¬ 
tiles, like salamanders, for example, such as the lim¬ 
bic cortex, which we will discuss later in this chapter. 
That region has five cortical layers and is sometimes 
referred to as 'paleocortex'. 
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The six horizontal layers of cortex are organized in 
cortical columns , verticalbarrel-shaped slices (Figure 5.8). 
These often contain closely related neurons, such as 
visual cells that respond to different orientations of 
a single light edge in just one part of the visual field. 
Columns may be clustered into hyper columns, which 
may be part of an even larger cluster. Thus, cortex has 
both a horizontal organization into six layers, and a 
vertical one, into columns, hypercolumns, and eventu¬ 
ally entire specialized regions. The visual cortex of the 
macaque monkey is often used as an animal model to 
study vision. Human visual cortex looks quite simi¬ 
lar. Note that there are six layers, with numbering (in 
Roman numerals) beginning at the top with layer I 
and progressing down to layer VI (Figure 5.9). 


The geography analogy is useful, but the brain, like 
the world, is a dynamic place. New streets are built 
and old ones move or are rebuilt. Houses and their 
residents appear and disappear. Until about a decade 
ago, it was believed that neurons did not change in the 
adult brain, but we now know of a number of ways 
in which neurons continue to grow, migrate, connect, 
disconnect, and die, even in the healthy mature brain. 
The brain is never frozen into a static rock-like state. 

These dynamic aspects of the brain can be seen 
even at the level of the six layers of cortex. Let's take 
another look at the six layers of the cortex, this time 
using a schematic drawing of the layers and their 
inputs and outputs (Figure 5.9). Notice that some cor¬ 
tical neurons send their axons to the thalamus, while 


Cortical 
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FIGURE 5.9 A schematic drawing of the six layers of cortex, the gray matter. Note that some cortical neu¬ 
rons send their axons to the thalamus, while others receive input from thalamic neurons. Ipsilateral = same side 
of the cortex; Contralateral = opposite side. 
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others receive input from thalamic neurons. Millions 
of cortical nerve cells go to the opposite hemisphere 
(the contralateral hemisphere), while many others 
project their axons to the same hemisphere (the ipsi- 
lateral side). However, the densest connections are to 
neighboring neurons. Cortical layer I consists largely 
of dendrites (input fibers) that are so densely packed 
and interconnected that this layer is sometimes called 
a 'feltwork', a woven sheet of dendrites. The neurons 
in this drawing (Figure 5.9) are called 'pyramidal' 
because their bodies look like microscopic pyramids. 
They are embedded in a matrix of glial cells, which 
are not shown here. These connection patterns in cor¬ 
tex undergo major change in human development 
and through-out the lifespan (see Chapter 15 for more 
discussion of this). 


2.0 GROWING A BRAIN FROM THE 
BOTTOM UP 

2.1 Evolution and personal history are 
expressed in the brain 

We usually see the brain from the outside, so that the 
cortex is the most visible structure. But the brain grew 
and evolved from the inside out, very much like a 
tree, beginning from a single seed, then turning into 
a thin shoot, and then mushrooming in three direc¬ 
tions: upward, forward, and outward from the axis of 
growth. That point applies both to phylogenesis - how 
species evolved - and ontogenesis - how the human 
brain grows from the fetus onward. 



Fluid ventricles 
(light blue) 









Hippocampi 
(light brown) 


FIGURE 5.10 Growing the brain from the bottom up. If you can memorize these basic 
shapes, you will have a solid framework for understanding the brain. Notice how the brain builds 
on the brainstem, with the thalami on top as major input hub. The hippocampi and amygdalas are 
actually nestled inside each of the temporal lobes. The light blue fluid ventricles have no neurons, 
but provide the brain's own circulatory system. The basal ganglia can be thought of as the output 
hub of the system. A great deal of traffic flows back to the cortex as well. Source : Baars and Fu, with 
permission. 






134 


5. THE BRAIN 



FIGURE 5.11 Diagram of the evolution of the mammalian 
brain. The forebrain evolves and expands along the lines of the 
three basic neural assemblies that anatomically and biochemically 
reflect ancestral commonalities with reptiles, early mammals, and 
late mammals. Source : Adapted from MacLean, 1967, with kind 
permission. 

The mature brain reveals that pattern of growth 
and evolution. It means, for example, that lower 
regions like the brainstem are generally more ancient 
than higher regions, such as the frontal cortex. Basic 
survival functions like breathing are controlled by 
neural centers in the lower brainstem, while the large 
prefrontal cortex in humans is a late addition to the 
basic mammalian brain plan. It is located the farthest 
upward and forward in the neural axis (Figure 5.11). 
Thus, local damage to prefrontal cortex has little 
impact on basic survival functions, but it can impair 
sophisticated abilities like decision-making, self- 
control, and even personality. 

2.2 Building a brain from bottom to top 

Because the brain involves hundreds of millions of 
years of evolutionary layering on top of older layers, 
the more recent levels hide the older ones. That is par¬ 
ticularly true for the fast-ballooning neocortex in pri¬ 
mates and humans, called the 'new cortex' because 
it is more recent, and has six layers rather than the 
four or five layers of the reptilian and early mamma¬ 
lian brain. The brain therefore grows a little bit like a 



FIGURE 5.12 Do you really need a cortex? A structural brain 
scan (MRI) from a 7-year-old girl who had a surgical removal of her 
left hemisphere at age 3 for Rasmussen's encephalitis. Such surger¬ 
ies can save children's lives if they are performed early enough. 
Because the brain is highly flexible at this age, the language capac¬ 
ity has shifted to the right hemisphere. Notice, however, that her 
brainstem and thalami are intact. The brainstem is crucial to life 
functions, and cannot be removed. She is able to play and talk, 
and has mild right side motor impairment. Source : Borgstein and 
Grotendorst, 2002. 


mushroom over the course of evolutionary time. The 
neuraxis - the spinal cord and brain - grows from tiny 
cellular clumps, then forward into a slender cylindri¬ 
cal shoot, and then thickening centrifugally to form 
the spinal cord, covered by an approximate mush¬ 
room shape. In the womb, the embryonic brain devel¬ 
ops into an S shape, and then the neocortex covers the 
older regions. 

We can follow the brain from bottom to top to show 
structures that are normally hidden by the head of the 
mushroom. We encourage you to draw these succes¬ 
sive levels of the great tower of the brain (Figure 5.10). 

Unlike most other mammals, humans stand 
upright, and therefore bend their eyes and cortices at 
a right angle forward. That is why the upper direc¬ 
tion of the human brain is both called 'dorsal', mean¬ 
ing 'toward the back' and also 'superior', meaning 
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Dorsal 

cochlear 


FIGURE 5.13 Detailed anatomy of the brainstem and pons. Notice that all the major input-output pathways of the brain 
emerge here, either flowing down the spinal cord, or out through narrow openings in the cranium. Vision, hearing, olfaction, and 
taste use cranial nerves as major pathways. Touch and pain perception in the head do the same. The brainstem also controls vital 
functions like breathing and heart rate. (Afferent = input to cortex; efferent = output from cortex). Source: Standring, 2005. 


upward. The other directions are called 'ventral', 
'toward the belly', and also 'inferior', meaning down¬ 
ward. We have a double vocabulary for the human 
brain, an important point to understand in order to 
avoid getting confused. 

In this section, we will 'grow' a brain, beginning 
at the bottom with the older regions of the brain and 
layering on until we come to the newest part of the 
brain, the neocortex. We begin with the brainstem and 
pons which are at the bottom or 'oldest' section of the 
brain. 

The brainstem (Figure 5.13) is continuous with the 
spinal cord. Its upper section, the pons, has nerve 
fibers that connect the two halves of the cerebellum. 
The brainstem and pons form a major route from the 
spinal cord to the brain. Some basic functions such as 
control of breathing and heart rate are controlled here. 


Next, we have the thalamus - actually, they are the 
thalami, because there are two of them, one in each 
hemisphere (Figure 5.14). The two egg-shaped thalami 
form the upper end of the brainstem. The thalami are 
the great traffic hubs of the brain. They are also inti¬ 
mately connected with each great hemisphere. 

Immediately below and in front of each thala¬ 
mus is a cluster of nuclei called the hypothalamus. It is 
connected with the pituitary gland, often called the 
'master gland' of the body (Figure 5.15). Together, 
the hypothalamus and pituitary are an extraordinar¬ 
ily important neurohormonal complex. Many types 
of physiological homeostasis are monitored by the 
hypothalamus. When hypothalamic neurons detect a 
deviation from the proper blood level of oxygen, they 
trigger increased breathing - such as the sigh you 
might make after reading intensively in a cramped 
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FIGURE 5.14 Transparent overview of the thalamus in the 
center of each hemisphere, and the basal ganglia looking like a "shield 
and loop' on the outer side of each thalamus. Source: Ohye, 2002. 


position. The hypothalamus also has crucial emotional 
functions. 

Seated on top of the thalami like a rider on a horse are 
the two hippocampi, one on each side (Figures 5.10,5.17). 
Each hippocampus is nestled inside of a temporal lobe 
on each side, as we will see later on. But it is impor¬ 
tant to see the doubled structure of two hippocampi. 
As we have seen, the hippocampus plays a major role 
in transferring experiential information to longer-term 
memory, and in retrieving episodic memories as well. 
It is also involved in spatial navigation. 

Near the tip of each hippocampal loop is an 
almond-shaped structure called the amygdala, which 
plays a starring role in emotions and emotional asso¬ 
ciation (Figure 5.18). 

The next level up is deceiving. It looks like a neu¬ 
ral structure but is not. It is the four ventricles, of which 
you can see the right and left one (Figure 5.10). The 
ventricles are small cavities containing a circulat¬ 
ing fluid that is separate from the bloodstream. This 
brain-dedicated circulatory system descends into the 
spinal cord through a tiny tube called the aqueduct, 
and the fluid of the ventricular system is therefore 
called the cerebrospinal fluid. The ventricular walls 
have recently been found to be sites for neural stem 
cells, much to the surprise of many scientists. It was 
long believed that neurons could not be replaced dur¬ 
ing life, but certain regions like the hippocampus and 


olfactory surface replace their cells, just as the rest of 
the body does. The ventricular stem cells are believed 
to be a source of these new neurons. 

Next up are the basal ganglia, literally, the clumps 
at the bottom of the brain (Figure 5.19). There is one 
outside of each thalamus. The elegant shield-like 
structure with outward radiating tubes is called the 
putamen. Looping over each is another artistic struc¬ 
ture called the caudate nucleus. This 'shield and loop' 
structure is fundamentally important for control of 
movement and cognition. Notice that the basal ganglia 
are located outside of the thalami. 

Finally, we can mount the two hemispheres on 
top of the lower levels of the brainstem, thalami, hip¬ 
pocampi and amygdala, ventricles and basal ganglia 
(Figure 5.20). So you should not be surprised when 
you carve away the cortex to see deeply buried, more 
ancient brain structures appear in the excavation. 

One final note on 'growing' the brain: we present a 
bottom view of the brain in order to show you some 
brain regions that are difficult to see otherwise (Figure 
5.21). You will notice the optic nerve linking the eyes, 
for example, to the cortex. 

So there you have the brain, shown 'growing' from 
ancient areas to the neocortex in the two hemispheres. 
Now let's take a look at the functional significance of 
these brain areas in human cognition. In this discus¬ 
sion, we will proceed in a 'top down' fashion, begin¬ 
ning with the two hemispheres, moving through the 
major lobes, and then on to the subcortical 'satellites' 
of the brain. 


3*0 FROM 'WHERE' TO 'WHAT': THE 
FUNCTIONAL ROLES OF BRAIN 
REGIONS 


We have discussed the many levels of analysis with 
which to understand brain structure and shown where 
the major brain areas are located. Let's work through 
the brain now, beginning with the neocortex and end¬ 
ing with the brainstem, and discuss the functional 
roles they play in human cognition. 


3*1 The cerebral hemispheres: the 
left-right division 

The two mirror-image halves of the cortex have puz¬ 
zled people for centuries. Why are there two hemi¬ 
spheres? If we have but one mind, why do we have 










FIGURE 5.15 The hypothalamus (highlighted with dark blue circle) is a cluster of nuclei located immediately below and in front of each thalamus. The 
hypothalamus is important in regulating physiological and emotional processes and is closely connected with the pituitary gland. Source : Standring et ah, 
Gray's Anatomy , 2008, Chapter 21, Figure 21.11. 
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FIGURE 5.16 We begin 'growing' the brain with the brain¬ 
stem and pons. Source : Baars and Fu. 



FIGURE 5.18 The amygdalas are situated just in front of the 
tip of each hippocampus. Source : Baars and Fu. 



Anterior limb of 



FIGURE 5.19 Side view of the basal ganglia, with the 'shield 
FIGURE 5.17 Schematic drawing of the hippocampi. Source : and loop' formed by the putamen and caudate nucleus, respectively. 

Baars and Fu. Source : Standring, 2005. 
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FIGURE 5.20 The cerebral hemispheres are shown mounted 
above the brainstem and other subcortical bodies. Source : Baars and Fu. 



FIGURE 5.21 A view of the brain from below showing the 
medial temporal lobe and optic tracts. Source : Baars and Fu. 


two hemispheres? Sir Charles Sherrington (1947) 
wrote: 

This self is a unity ... it regards itself as one, oth¬ 
ers treat it as one. It is addressed as one, by a name to 
which it answers. The Law and the State schedule it as 
one. It and they identify it with a body which is consid¬ 
ered by it and them to belong to it integrally. In short, 
unchallenged and unargued conviction assumes it to be 
one. The logic of grammar endorses this by a pronoun 
in the singular. All its diversity is merged in oneness. 

The philosopher Rene Descartes, for example, was 
dumbfounded by the doubled nature of the brain. 



FIGURE 5.22 A cut-away of a three-dimensional magnetic 
resonance image showing the location of the corpus callosum - a 
white fiber arch extending horizontally from the anterior of the brain 
to the posterior, forming a fiber link between the two hemispheres. 
Source : Mark Dow, University of Oregon, with permission. 

Because he believed that the soul must be a unitary 
whole, he looked for at least one brain structure that 
was not doubled, and finally decided on the small 
pineal gland at the back of the brainstem. There he 
believed the soul resided - roughly what we mean by 
subjective experience. Unfortunately for Descartes, 
when microscopes became powerful enough to exam¬ 
ine the tiny pineal gland in detail, it also turned out to 
have two symmetrical halves, roughly mirror images 
of each other. 

How do the two hemispheres Talk' to each other? 
The answer lies in the fiber tract that runs from the front 
to the back of the brain, linking the two hemispheres. 

3.1.1 The corpus callosum 

The hemispheres are completely separate, divided 
by the longitudinal fissure that runs between the two 
hemispheres from the anterior (front) to the posterior 
(back) part of the brain. The link between the hemi¬ 
spheres is provided by the corpus callosum , a large arch 
of white matter (Figure 5.22). The number of axons 
traveling between the two hemispheres is estimated 
at more than 100 million. The corpus callosum has fib¬ 
ers that project between the hemispheres in an orderly 
way, with regions in the anterior portion connecting 
similar brain areas in the frontal lobes and regions in 
the posterior portion connecting similar brain areas in 
the occipital lobe. 
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FIGURE 5.23 A top view of the two hemispheres. Schematic drawing of the two halves of the cerebral cortex, showing 
some major functions of the right and left hemispheres. Note the massive bridge of the corpus callosum connecting the two 
sides. The eyes on top focus on converging lines in order to enable stereoscopic depth perception. Source: Standring, 2005. 


The role of the two hemispheres in human cogni¬ 
tion and the mind-brain has been the subject of exten¬ 
sive study, and we are still unfolding the subtle and 
not-so-subtle differences in the roles that the mir¬ 
ror-image hemispheres play in perception, language, 
thought, and consciousness. There are some hemi¬ 
spheric differences that are fairly well understood, 
such as crossover wiring. Many aspects of sensory 
and motor processing entail the crossing over of input 
(sensory) or output (motor) information from the left 
side to the right, and vice versa (Figure 5.23). 

For example, each optic nerve coming from the retina 
is split into a nasal half (on each side of the nose), which 
crosses over to the opposite side of the brain, and a lat¬ 
eral half, which proceeds to the same side (ipsilaterally). 
Only the olfactory nerve, which is a very ancient sen¬ 
sory system, stays on the same side of the brain on its 
way to cortex. The cortical output control of the hands 


is also crossed over, with the left hemisphere controlling 
the right hand, and the right controlling the left hand 
(Figure 5.24). While the left and right hemispheres have 
some different functions, the corpus callosum has some 
100 million fibers, constantly trafficking back and forth, 
which serves to integrate information from both sides. 
The time lag between the two hemispheres working 
on the same task may be as short as 10 ms, or one-hun¬ 
dredth of a second (Handy et ah, 2003). Therefore, when 
the great information highway of the corpus callosum is 
intact, the differences between the hemispheres are not 
very obvious. But when it is cut, and the proper experi¬ 
mental controls are used to separate the input of the 
right and left half of each eye's visual field, suddenly 
major hemispheric differences become observable. 

The question of the perceived unity of the world 
continues to interest scientists. The most spectacular 
finding in that respect has been the discovery that the 


















FIGURE 5.24 The pattern of cortical control over regions of the body. Notice that sensation and cortical motor control pathways cross 
over in the brain. Simple reflexes do not cross over, and coordination involves interaction between both sides. Source: Standring, 2005. 


corpus callosum can be cut in humans, without chang¬ 
ing the perceived unity of the world and of the self. 
Indeed, for many years such callosectomies (separa¬ 
tion or cutting the corpus callosum) were performed to 
improve uncontrollable epilepsy A complete slicing of 
the corpus callosum is called a callosotomy; more com¬ 
mon, today, is a partial cut of only the regions of the 
two hemispheres that spread epileptic seizure activity 
This partial cut is called a callosectomy. Doctors and 


their patients believed that cutting some 100 million fib¬ 
ers in the corpus callosum had no noticeable effect at all! 
It is a dramatic illustration of the capacity of the brain 
to adapt to quite severe damage - to fill in the missing 
details of the experienced world by means of eye move¬ 
ments, for example. More careful study, however, has 
provided evidence that a complete slicing, a callosot¬ 
omy does have subtle but long-lasting effects and so a 
partial resection, or callosectomy, is preferred. 


FRONTIERS OF COGNITIVE NEUROSCIENCE 


Synesthesia 



FIGURE 5.25 David Eagleman, PhD, Department of 
Neuroscience, Baylor College of Medicine, Houston, Texas, USA 


Imagine a world of magenta Tuesdays, tastes that have 
shapes, and wavy green symphonies. One in a hun¬ 
dred otherwise ordinary people experience the world 
this way, in a condition called synesthesia - the fusion 
of different sense experiences. In synesthesia, stimula¬ 
tion of one sense triggers an experience in a different 
sense. For example, a voice or the sound of music are 
not just heard but also seen, tasted, or felt as a touch. 
Synesthesia is a fusion of different sensory perceptions: 
the feel of sandpaper might evoke the musical sound of 
F-sharp, a symphony might be experienced in blue and 
gold colors, or the concept of February might be experi¬ 
enced as being located above one's right shoulder. Most 
synesthetes (people with synesthesia) are unaware that 
their experiences are in any way unusual. 
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Synesthesia comes in many varieties, and having one 
type gives you a high chance of having a second or third 
type. Experiencing the days of the week in color is the 
most common kind of synesthesia, followed by colored 
letters and numbers. Other common varieties include 
tasted words, colored hearing, number-lines perceived 
in three dimensions, and the personification of letters 
and numerals. 

Synesthetic perceptions are involuntary, automatic, 
and generally consistent over time. Moreover, synes¬ 
thetic perceptions are typically intrinsic, meaning that 
what is sensed is something like a simple color, shape, 
or texture, rather than something that is a thought asso¬ 
ciation. Synesthetes don't say, "This music makes me 
experience a vase of flowers on a restaurant table." It 
just happens to them. 

Synesthesia seems to be the result of increased cross¬ 
talk among sensory areas in the brain - like neighbor¬ 
ing countries with porous borders on the brain's map. 
Synesthesia has fascinated laypersons and scientists 
alike with its wealth of sensory amalgamations, but only 
recently has it been appreciated how the brains of such 
individuals yield surprising insights into normal brain 
function. 

Although synesthesia has been explored in behav¬ 
ioral and neuroimaging experiments, its genetic basis 
remains unknown. My laboratory group realized that 
synesthesia is an ideal condition for genetic analysis, 
for three reasons: (1) synesthesia clusters in families 
and appears to be inherited; (2) synesthetic perception 
results from increased cross-talk between neural areas, 
which suggests a set of candidate genes; and (3) a bat¬ 
tery of tests developed in our lab allows for confident 
identification of real synesthetes, not just people who 
have free associations to their experiences. 

We therefore are performing a large-scale genetic 
study, called a family linkage analysis, to map the 
gene(s) that correlate with color synesthesias. To this 
end, we have developed a battery of tests to clearly iden¬ 
tify synesthetes; that is, to distinguish them from control 
subjects. These tests are offered free to the research com¬ 
munity at www.synesthete.org. Several families with 
multiple synesthetes have provided pedigrees, and we 
have harvested DNA samples from over 100 people in 
these families. A genome wide scan is identifying the 
most probable genetic region responsible for synesthesia 
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FIGURE 5.26 in a common form of synesthesia, months 
and days of the month can have both colors and specific spa¬ 
tial configurations. Source : Cytowic and Eagleman, 2009 in 
Wednesday is Indigo Blue: Discovering the Brain of Synesthesia. 
Cambridge: MIT Press. 


in these families. Understanding the genetic basis of 
synesthesia yields insight into the way normal brains 
are wired. And it demonstrates that more than one kind 
of brain - and one kind of mind - is possible. 

Synesthesia affects the brain wiring of one in several 
hundred people, making it far more common than origi¬ 
nally thought, and far more important scientifically than 
a mere curiosity. Other evidence suggests that we may 
all be synesthetic to some extent - but the majority of us 
remains unconscious of the sensory fusions going on in 
our brains. 
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FIGURE 5.27 A view of functional areas 
in some of the sensory regions of the cortex. 
The central sulcus is seen separating the fron¬ 
tal lobe from the parietal lobe. Immediately 
posterior to the central sulcus is the primary 
somatosensory area. The Sylvian fissure is also 
called the lateral fissure. Source : Squire et al, 
2008. 


3.2 Output and input: the front-back division 

The cortex is a folded sheet of gray matter that would 
measure roughly 2 feet by 2 feet (60 cm by 60 cm) if 
it were unfolded. To fit within the skull, the cortex is 
folded into hills (gyri) and valleys (sulci). The cortex 
contains four lobes that are visible from the outside 
and two large regions that are not visible. Before we 
discuss the functions of these regions, let's take a look 
at another major division of the brain: the front-back 
division of the cerebral cortex. In order to understand 
this division, you will need to be able to locate some 
landmarks in the brain. In Figure 5.27, see if you can 
locate the central sulcus that runs vertically between 
the frontal lobe and the parietal lobe. To locate it, look 
for the region labeled 'primary somatosensory'. The 
central sulcus is just in front of, or anterior to, this 
region. The second landmark to look for is the Sylvian 
fissure. It runs more or less horizontally from the fron¬ 
tal lobe posterior, separating the temporal lobe from 
the parietal and frontal lobes. 

The sensory - or input - regions of the cortex are 
located posterior to the central sulcus and the Sylvian 
fissure, in the parietal, temporal, and occipital lobes. 
These lobes contain the visual cortex, auditory cortex, 
and somatosensory cortex, where information coming 
from the eyes, ears, and body is processed. The visual 
cortex, for example, begins in the occipital lobe but 


extends to the parietal and temporal lobes. The audi¬ 
tory cortex is located in the temporal lobe but also 
extends to the parietal lobe. Somatosensory areas are 
located in the parietal lobe. Taste and smell regions are 
located at the bottom of the temporal lobes. This 'back 
of the brain' large region, encompassing three cortical 
lobes, is not simply a site for processing sensory infor¬ 
mation. It is also the region of cortex for associative 
processes, where information from the various senses 
is 'bound together' for higher order processing. Think 
about watching a movie - these association areas will 
help you understand how to relate what you are hear¬ 
ing to what you are seeing on the screen. Much of this 
type of processing occurs in the parietal lobe, and we 
will discuss this important lobe in more detail in the 
next section. These association regions are largest in 
primates and largest of all in humans. 

The motor - or output - regions of the cortex are 
located in the frontal lobe, anterior to the central sul¬ 
cus and the Sylvian fissure. Look again at Figure 5.27 
and locate the region labeled 'primary somatosen¬ 
sory', just posterior to the central sulcus. Although it 
is not labeled on this figure, the primary motor region 
is in the frontal lobe, just across the central sulcus and 
anterior to the somatosensory regions in the pari¬ 
etal lobe. The close physical connection between the 
somatosensory cortex and the motor cortex allows for 
a tight coupling between the senses of touch, pressure. 
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FIGURE 5.28 Drawing of the somatosensory homunculus, 
showing the representation of body areas in the cortex. Note that some 
body areas, such as the face, have a disproportionately larger repre¬ 
sentation than other areas, such as the trunk. Source: Standring, 2005. 


and pain and the action or motor system. In fact, there 
is an intricate mapping of the body that is reflected in 
similar ways in the somatosensory region located just 
posterior to the central sulcus and its corresponding 
motor region located just anterior. This intricate map¬ 
ping is a representation of areas of the body: the dif¬ 
ferent regions of the body are not equally represented 
in these cortical regions; some areas, such as the face 
and hands, have quite a disproportionately large rep¬ 
resentation and other regions, such as the center of the 
back, have a disproportionately small representation. 
Consider how much more sensitive your fingertips 
are to touch, pressure, and pain than, say, the small of 
your back. The representational map in cortex reflects 
this differing sensitivity. There are two maps of the 
body: one is in somatosensory cortex and a very simi¬ 
lar one is in motor cortex (Figures 5.28 and 5.29). 

These two body maps or homunculi (Tittle men') 
were first discovered by the pioneer neurosurgeon 
Wilder Penfield at the University of Montreal in the 
1950s and 1960s. Penfield's team was the first to stim¬ 
ulate the cortex of awake patients, which is possible 
because the cortical surface contains no pain receptors. 
Therefore, local anesthetic applied to the incision was 
enough to dull the pain of the removal of the scalp. 



FIGURE 5.29 Drawing of the motor homunculus, showing the 
representation of body areas in motor cortex. Note that some body 
areas, such as the face, have a disproportionately larger representa¬ 
tion than other areas, such as the trunk. Source: Standring, 2005. 

and surgeons could electrically stimulate the exposed 
cortical surface and ask their awake patients about 
their experiences as a result. Their discoveries have 
largely stood the test of time. Exploration by electri¬ 
cal stimulation was medically necessary in order to 
know where to operate in the brain while minimizing 
damage to functional regions in patients. In the case 
of the sensory homunculus (somatosensory), local 
stimulation would evoke feelings of touch in the cor¬ 
responding part of the body. Stimulation of the motor 
homunculus would evoke specific body movements, 
but interestingly, patients would deny a sense of own¬ 
ership of those movements. When Penfield would ask, 
'Are you moving your hand?' when stimulating the 
hand region, a patient might say, 'No, doctor, you're 
moving my hand'. If, however, the surgeon moved 
perhaps a centimeter forward to the pre-motor strip, 
stimulation would evoke a reported intention to move 
one's body, without a sense of being externally con¬ 
trolled. It is a fundamental distinction, which we will 
return to later. 

The essential point here is that the central sulcus 
is an important landmark to know. Not only does it 
separate the sensory and motor homunculi but, more 
broadly, the central sulcus separates the more sensory 
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half of cortex (posterior), from the frontal half (ante¬ 
rior). Posterior cortex contains the projection regions of 
the major sense organs - vision, hearing, touch, smell, 
taste. In contrast, frontal cortex is involved in action 
control, planning, some working memory functions, 
language production, and the like. In a sense, the pos¬ 
terior half deals with the perceptual present, while the 
anterior half tries to predict and control the future. 


3*3 The major lobes: visible and hidden 

We have used the analogy of the geography of the 
brain. In this setting, the major lobes can be viewed as 
large continents in brain geography. While each is sep¬ 
arate from the other and has its own local functions 
and anatomical features, each is part of the whole, the 
brain, and thus is united and intimately linked to total 
brain function. The four 'continents' of the brain are 
shown in Figure 5.30 and include the frontal, parietal, 
temporal, and occipital lobes. In this section, we will 
discuss their functional roles in cognition. Two other 
major regions, not visible from the exterior view of the 
brain, play important roles in cognition and we will 
describe those as well. 


3 . 3 .1 Frontal lobe 

The massive frontal lobe is the site for motor planning 
and motor output. As we mentioned, the motor areas 
are tightly connected to the somatosensory regions 
with similar homunculus maps representing body 
areas. These motor functions that are present in the 
human brain are present in most mammalian brains 
in a similar way. But the frontal lobe in humans is far 
larger than in non-human primates or any other crea¬ 
ture. What other functions does the frontal lobe per¬ 
form and how is its role unique in humans? 

The frontal lobe has been termed the 'organ of civi¬ 
lization' (Luria, 1966). The regions of the frontal lobe 
that have earned this term are primarily in the pre¬ 
frontal cortex. The prefrontal cortex is located on the 
medial, lateral, and orbital surfaces of the most ante¬ 
rior portion of the frontal lobe (Figure 5.31). 

Prefrontal cortex is the non-motor part of frontal cor¬ 
tex. Notice that prefrontal cortex is the most forward 
part of the frontal cortex. The term 'prefrontal' is some¬ 
what confusing, but it means 'at the front of the fron¬ 
tal cortex'. There are no obvious boundary markers for 
prefrontal cortex, which is defined instead by a set of 
projections from the thalamus. Nevertheless, prefron¬ 
tal cortex is perhaps the most distinctively 'cognitive' 



FIGURE 5.30 Basic brain directions. Because the human brain 
is rotated 90 degrees forward from the spinal cord (unlike most mam¬ 
mals and reptiles), it has two sets of labels. The dorsal direction is also 
called superior, the ventral is also called inferior, and rostral, roughly 
the same as frontal and caudal, is sometimes called posterior. To 
simplify, just use plain language, like front, back, upper, and lower. 
Source : Standring, 2005. 


part of the brain. The prefrontal cortex is a large cortical 
region, taking up an estimated one-third of the entire 
area of cortex. What is the prefrontal cortex specialized 
for and why is it so uniquely a human region? 

The prefrontal cortex is specifically needed for: 

• initiating activities 

• planning 

• holding critical information ready to use (an aspect 
of working memory) 

• changing mental set from one line of thinking to 
another 

• monitoring the effectiveness of one's actions 

• detecting and resolving conflicting plans for action 

• inhibiting plans and actions that are ineffective or 
self-defeating. 

This list shows how important the prefrontal cortex is 
to human cognition. Many anatomists believe that pre¬ 
frontal cortex is largest in humans, and distinguishes 
our species from other primates. In addition, the pre¬ 
frontal cortex has regions for emotional and person¬ 
ality processes as well as social cognition - knowing 
'how to behave' for example. On the lateral convexity, 
interposed between the dorsolateral prefrontal and the 
ventral portion of premotor cortex, is Broca's area. This 
area is involved in the abstract mediation of the verbal 
expression of language, a uniquely human function. 

The frontal lobe, then, is far larger in humans than 
other primates and has developed many new functions 
and processes for dealing with human activities such 
as language, thought, and executive control of higher 
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FIGURE 5.31 Left: An activation map rendered on a three-dimensional magnetic resonance image showing regions in the medial pre¬ 
frontal cortex. Right: How to find the prefrontal cortex. The entire frontal cortex is in front of the central sulcus, the vertical fold that runs 
from the top of the cortex down to the temporal lobe. Locate the central sulcus in this figure. The two purple gyri (hills) immediately in front 
of the central sulcus are called the motor and premotor cortex. The reddish-purple patch in front of that is called the supplementary motor 
cortex. However, the three shades of yellow in the frontal third of the whole cortex is prefrontal cortex, often considered the most 'cognitive' 
part of the brain. Source : Harenski and Hamann, 2006. 


order processes. A second lobe that has also evolved to 
be much larger in humans is the parietal lobe and we 
will see what functions it performs in the next section. 

3 . 3.2 Parietal lobe 

As we noted earlier, the anterior region of the pari¬ 
etal lobe holds the somatosensory cortex. However, 
the parietal lobe is not just a somatosensory region in 
humans, much as the frontal lobe is not just a motor 
region. One important function of the parietal lobe is 
multiple maps of body space. What does 'body space' 
mean exactly? Think about sitting in a chair at a table 
and looking down at your hands. Your eyes bring sen¬ 
sory input to your brain about where your hands are 
in respect to your body, but there are other inputs tell¬ 
ing you where you hands are as well (which is why 
you know where your hands are even if your eyes are 
closed). Your imagined hand position will be from 
your own perspective, or the egocentric perspective. 
Now imagine a friend sitting across the table from you 
and conjure up where your hands are from his or her 
perspective. How do you do this? It is easy to accom¬ 
plish and regions in the parietal lobe are where this 
type of processing take place (Figure 5.32). 

Posterior and inferior to the somatosensory region is 
an area termed the inferior parietal lobe or IPL. The func¬ 
tional significance of this region is still being elucidated. 


However, it is thought to be the site for multisensory 
integration. 

3 . 3.3 Temporal lobe 

The temporal lobe is the region where sound is pro¬ 
cessed and, not surprisingly, it is also a region where 
auditory language and speech comprehension systems 
are located. The auditory cortex is located on the upper 
banks of the temporal lobe and within the Sylvian fis¬ 
sure. Just posterior to the auditory cortex is Wernicke's 
area for speech comprehension. But the temporal lobe 
is not only a sound and language processing region. 
The middle sections of the temporal lobe are thought 
to contain conceptual representations for semantic 
knowledge. More inferior and posterior temporal lobe 
areas are more finely tuned for representing visual 
objects and include the fusiform face area. 

3 . 3.4 Occipital lobe 

The occipital lobe, at the very posterior region of cortex, 
is home to visual cortex. Most of visual cortex is hidden 
within the calcarine fissure. The visual system occupies 
a large area within the occipital lobe that extends ante¬ 
rior to the parietal and temporal lobes. New techniques 
provide the ability to 'inflate' these cortical regions to 
remove the folds and allow us to see the functional 
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FIGURE 5.32 Schematic of some of the multisensory functions of the parietal lobe. The sight 
and sound of the bell are combined by neurons in the parietal cortex, using a 'map' of the space sur¬ 
rounding the body (egocentric space). Source : Beauchamp, 2005. 



FIGURE 5.33 An actual human brain, showing the insula just 
above and hidden behind the temporal lobe. Source : Standring, 2005. 

visual regions that are normally tucked into the cal¬ 
carine fissure and difficult to see on a brain scan. 



FIGURE 5.34 A cut-away view of the left hemisphere reveal¬ 
ing the insula, which is not visible from a lateral view. Insula' 
means 'island' because of this appearance when the brain is dis¬ 
sected. Source : Standring, 2005. 


3 . 3.5 The insula and Sylvian fissure 

Like a large tree, the cortex has grown to cover up large 
parts of itself, as we can see by inflating the cortex 
mathematically and spreading it into a flat sheet. Two 
of the areas that are hidden by the expanding cortex are 


especially important: the insula and the Sylvian fissure. 
When the temporal lobe is gently pulled away from the 
rest of cortex, a new hidden world appears. This region 
is called the 'insula', or 'island', because it appears like 
a separate island of cortex (Figures 5.33 and 5.34). The 
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FIGURE 5.35 The medial temporal lobe (MTL) - the midline regions seen from the bottom. This is the ancient 
"smell brain' which is now surmounted by a massive 'new' cortex in higher mammals. It is therefore difficult to see 
from the outside, but it still retains many essential functions, including encoding conscious events into memories 
(episodic memories). Source : Buckley and Gaffen, 2006. 


insula is not often seen in popular accounts of the brain, 
but it involves hundreds of millions of neurons and 
quite a wide expanse of cortical surface. Neurological 
evidence suggests that it may be involved in 'gut feel¬ 
ings' like the conscious sense of nausea and disgust. But 
the insula is so large that it probably has multiple func¬ 
tions. There does seem to be good convergent evidence 
that interoception - feelings of one's inner organs - may 
be one of the major roles of the secret island of cortex. 

One researcher suggests that: 'In humans, . . . the 
right anterior insula, . . . seems to provide the basis for 
the subjective image of the material self as a feeling (sen¬ 
tient) entity, that is, emotional awareness' (Craig, 2005). 

The Sylvian fissure is a very large sulcus that runs 
in a roughly horizontal direction from the frontal lobe, 
between the parietal and temporal lobes, ending near 
the junction of the parietal, temporal, and occipital 
lobes. The anatomy of the fissure differs widely across 
individuals and also between hemispheres. Tucked 
inside the Sylvian fissure, on the upper banks of the 
superior temporal gyrus, is the supratemporal plane. 
This region is called a plane because is it a somewhat 
flat bank of cortex extending from the lateral surface 
into the medial regions. The supratemporal plane is 
home to primary and secondary auditory cortex as 
well as parts of Wernicke's area for speech comprehen¬ 
sion. The upper bank of the Sylvian fissure, adjacent 
to the parietal lobe and opposite the supratemporal 


plane, is home to somatosensory cortex that wraps 
around and under the top section of the fissure. 

3 . 3.6 The medial temporal lobe 

The medial temporal lobe (MTL) is actually part of the 
temporal lobe, but its function and anatomy dif¬ 
fer strikingly and it is typically referred to as a sepa¬ 
rate structure. The MTL is home to the hippocampi 
and related regions that are associated with memory 
functions (Figure 5.35). There are many regions in the 
MTL, including a region called the limbic area. The 
word 'limbus' means 'boundary', and true to its name, 
there is a great deal of debate about the proper bound¬ 
aries of this region. You will occasionally see the entire 
complex of hippocampus, amygdala, and limbic cor¬ 
tex being called the 'limbic system'. All these terms 
have their uses, and it is just important to be aware of 
what is intended. 

The upper arc is called the cingulate gyrus ('cingu¬ 
lum' means belt or sash as in 'cinch'), which is nestled 
between the corpus callosum and the cingulate sulcus 
(Figure 5.36). The front half of this region generally 
lights up in brain scans during tasks that involve con¬ 
flicting stimuli or responses, a very important aspect 
of executive function. In the classical Stroop effect, for 
example, there is a conflict between the color of words 
and the meaning of the same words. The front half of 
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FIGURE 5.36 The medial temporal lobe and cingulate gyms 
(green upper loop), seen from the midline section of the brain. The 
hippocampus is colored purple and amygdala orange. They are 
actually embedded inside of the temporal lobe. Source: Heimer and 
Van Hoesen, 2006. 


the cingulate is somehow involved in detecting or 
resolving such conflicting signals. 

The lower arc of the limbic lobe is originally a part 
of the smell brain, the rhinal cortex, and is therefore 
called the perirhinal cortex, ('peri-' means 'around' and 
'rhinal' means 'nose'). Recall that we stated earlier that 
not all cortex has six layers; only the giant mammalian 
cortex does, which is why it is called 'neocortex' (new 
cortex, because it only emerged 200 million years ago). 
Older regions of cortex are also found in reptiles, like 
salamanders, for example, such as the limbic cortex. 
This region has five cortical layers and is sometimes 
referred to as 'paleocortex'. It is often associated with 
emotion and memory and, in the case of the upper arc 
of the limbic region, with decision-making and the res¬ 
olution of competing impulses. In addition, the limbic 
cortex flows continuously into the hippocampus and 
amygdala, which are hidden inside the temporal lobe, 
and therefore invisible from the medial perspective. 
Recent research shows very close interaction between 
these ancient regions of cortex and episodic memory, 
i.e. memory for conscious experiences. This is the 
ancient reptilian brain, which is, however, still a vital 
center of activity in humans and other mammals. 

3 A The massive interconnectivity of the 
cortex and thalamus 

While the lobes may be thought of as the continents 
of the brain, their processes are nonetheless intricately 


intertwined not only with each other, but also with the 
satellites of the subcortex in the massively intercon¬ 
nected brain. 

Sprouting from the cells in the grayish layers of cor¬ 
tex are billions of axons, the output fibers from nerve 
cells, and dendrites, which provide electrical input to 
each cell. When white support cells, called the mye¬ 
lin, wrap around those fibers, they look like a white 
mass to the naked eye, and are therefore called the 
white matter. The whole giant structure of the cortex 
is shaped much like a superdome stadium, with two 
giant halves, each filled with billions of cables going 
in all directions, centered on a thalamic hub nestled 
in the middle of each hemisphere (Figure 5.37). The 
two cortical half-domes, with a thalamic traffic hub on 
each side, create an extraordinary biological structure. 
Think of the thalamus as a relay station: almost all 
input stops off at the thalamus on the way to cortex; 
almost all output also stops off at the thalamus, going 
out to the muscles and glands. 

Fibers emanating from cortical cells spread in 
every direction, flowing horizontally to neighboring 
cells, hanging in great bundles on their way to dis¬ 
tant regions of cortex, and converging downward on 
the great traffic hub, the thalamus, of each half of the 
cortex. In addition, hundreds of millions of axons flow 
crosswise, from one hemisphere to the other, creating 
white axon bridges called commissures (Figure 5.38). 
The largest crosswise fiber bridge is called the corpus 
callosum, or 'calloused body'. When the brain is sliced 
straight through the midline, you can see the corpus 
callosum as a curved white bow shape. The white 
color, again, comes from the myelin surrounding the 
cortical axons that form the great bridge connecting 
the two hemispheres. 

Finally, cortical sensory and motor pathways 
make up the incoming and outgoing highways of the 
brain (Figure 5.39). All of these pathways flow from 
the bottom of the brain. The sensory and motor path¬ 
ways can be divided into two sets. One set of path¬ 
ways emerge through small holes in the cranium, 
the upper skull, and are therefore called the cranial 
nerves. These include the optic nerve from the back 
of the eyes, the auditory, olfactory, and taste nerves, 
as well as the feelings of touch and pain from the face 
and oral cavity; on the motor side, our facial expres¬ 
sions, vocal apparatus, and mouth, tongue, and so 
on are also controlled by cranial nerves. The second 
set of pathways flows into the spinal cord, and con¬ 
trols all our bodily functions, both voluntary - like 
movements of the torso, arms and legs - and vegeta¬ 
tive (autonomic), like blood pressure and sweating. 
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FIGURE 5.37 Cortex and thalamus: a single unified system. A schematic drawing showing a color-coded mapping of connections from 
the thalamus to cortical regions. Source : Standring, 2005. 


On the input side, sensory nerves from the body give 
us all the information, both conscious and uncon¬ 
scious, that we receive from the body While these 
pathways are complex in detail, the overview is 
straightforward. 

It is conventional to put an '-o-' between the names 
of brain regions that are connected, so that we can 


speak of the 'thalam-o-cortical' connections. Signal 
flow from cortex to thalamus is called corticothalamic, 
and, believe it or not, neuronal traffic can even be 
cortico-thalamo-cortical. It's a little less complicated 
if you think about it as the traffic flow in a city, or 
even as the World Wide Web, connecting millions of 
computers by way of major hubs and pathways.) 
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FIGURE 5.38 Schematic drawing of the connectivity of the 
brain, showing major fiber patterns. Source : Standring, 2005. 



FIGURE 5.39 White bundles of myelinated axons run in all 
directions through the cortical domes. Source: Mario Lazar, with 
kind permission. 


3.5 The satellites of the subcortex 

Because the human cortex is so large, it covers impor¬ 
tant subcortical organs, which act as satellites to the 
cortex, constantly interacting with it. These subcortical 
structures don't look like the popular idea of a brain 
at all - they are giant clusters of neurons often called 
'ganglia' or 'nuclei'. Subcortical organs often have 


remarkably elegant shapes, like loops, horns, and egg¬ 
like ovals. 

The satellite regions are especially important in 
cognitive neuroscience. The thalamus, often called 
the gateway to cortex, was described above; the two 
thalami reside at the very center of the brain, on both 
sides of the midline (so you can't actually see them 
in the medial view). The thalami also connect differ¬ 
ing cortical regions, so there are important cortico- 
thalamo-cortical circuits that have been shown to play 
a role in attentional processing and other higher order 
cognition functions. The thalami are nestled above the 
brainstem and just below cortex, a perfect location to 
serve their role as the relay station for the brain. 

The hippocampal complex (see Figure 5.17) is critical 
to remembering conscious experiences, and appears 
as two small sausages embedded in each temporal 
cortex. However, it is now known that areas adjacent 
to the 'sausage' of hippocampus are also needed for 
episodic (experiential) memory. For that reason we 
will talk about the entire hippocampal complex, rather 
than just the hippocampus alone. 

At the very front tip of each hippocampus is the 
amygdala, Latin for 'almond' (see Figure 5.18). It has 
a small spherical nut-like shape, profoundly impor¬ 
tant for emotions like fear and anger, as well as learn¬ 
ing processes that involve those emotions. Finally, the 
basal ganglia (see Figure 5.19) are complex disk-and- 
loop structures just outside of each thalamus, and 
the cerebellum (or little brain) rides on the back of the 
entire upper brainstem and thalami. The basal ganglia 
have been implicated in action planning and uncon¬ 
scious cognitive operations. New evidence, however, 
has linked the basal ganglia to higher order cognitive 
functions, such as decoding the grammar, or syntax, of 
language. 

The cerebellum is seated on the rear of the lower 
brainstem. It is itself a very large structure. In many 
mammals, the cerebellum has as many neurons as the 
cortex itself, though they have shorter axons. Most 
cerebellar neurons are connected locally, in small clus¬ 
ters. Historically, the cerebellum was thought to be 
mainly involved in controlling fine motor movements, 
like the finger movements of a typist or musician. It 
is now also known to be necessary for cognitive func¬ 
tions as well. Indeed, functional imaging shows the 
cerebellum to Tight up' in almost any cognitive task. 
The reason for this is not completely understood. 

Finally, a number of tiny nuclei of the brainstem 
and basal forebrain send cell fibers widely through 
the upper brain. These neuromodulating nuclei are 
sometimes informally called 'spritzers', because they 
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spray neurochemicals from their axon terminals so 
that those chemicals are widely dispersed. Spritzers 
may contain only a few thousand neurons, but they 
are crucial to a healthy brain. Major disorders like 
Parkinson's disease, characterized by disabling motor 
tremor, result from defects of such neuromodulators. 
They also control the daily sleep-waking cycle. 

We end this section with a description of the retic¬ 
ular formation, located at a central point in the brain 
(Figure 5.40). This is a particularly intriguing area of 
the brain in terms of its role in human conscious expe¬ 
rience. The reticular formation is called 'reticular' (i.e. 
network-like) because the neuronal axons in this sys¬ 
tem are usually very short, suggesting a great amount 
of interaction between adjacent neurons. Further, it 
receives input from all sensory and motor systems, 
as well as from other major structures in the brain. 
Through its connections with the thalamus, it can 
send information to, and receive it from, all areas of 
the cortex. 

What does this suggest about the role of the reticu¬ 
lar formation in conscious experience? There is neuro¬ 
physiological evidence that specialist systems in the 
brain can cooperate and compete for access to a cen¬ 
tral integrative 'blackboard'. There is reason to think 
that the extended reticular-thalamic system (ERTAS) cor¬ 
responds to this 'blackboard'. 

This is not a new notion; Aristotle's 'common 
sense' was supposed to be a domain of integration 



FIGURE 5.40 The ascending reticular activating system 
(ARAS) is found in the brainstem and thalamus, and sends projec¬ 
tions throughout cortex. The ARAS is thought to be required for the 
normal conscious waking state. Source : Filley, 2002. 


between the different senses. In fact, anatomists who 
have studied the reticular formation have pointed to 
its resemblance to Aristotle's concept. Scheibel and 
Scheibel (1965) point out that 'Anatomical studies of 
Kohnstamm and Quensel, which suggested pooling 
of a number of afferent and efferent systems upon the 
reticular core, led them to propose this area as a "cen¬ 
trum receptorium 2" or "sensorium commune" - a 
common sensory pool for the neuraxis'. 

Moreover, these authors note that '. . . the reticu¬ 
lar core mediates specific delimitation of the focus of 
consciousness with concordant suppression of those 
sensory inputs that have been temporarily relegated 
to a sensory role' (p. 579). Along similar lines, Gastaut 
(1958) describes the brainstem reticular formation as 
an area of 'convergence . . . where signals are concen¬ 
trated before being redistributed in a divergent way to 
the cortex'. Thus, different sensory contents can sup¬ 
press each other, as we would indeed expect of input 
to a global workspace. This suggests that different 
specialized processors can compete for access to the 
ERTAS. 

How does this 'blackboard' concept actually work 
in terms of neural processes and how are messages 
broadcast? In one possible scenario, one sensory 
projection area of the cortex provides input to the 
ERTAS. If this input prevails over competing inputs, 
it becomes a global message which is widely distrib¬ 
uted to other areas of the brain, including the rest 
of the cortex. Thus, one selected input to the ERTAS 
is amplified and broadcast at the expense of oth¬ 
ers. Thus, in this way, the ERTAS underlies the 'glo¬ 
bal broadcasting' function of consciousness, while a 
selected perceptual 'processor' in the cortex supplies 
the particular contents of consciousness which are to 
be broadcast. 

What is the role of the ERTAS in conscious thought? 
It may be the case that any cortical activity must trig¬ 
ger ERTAS 'support' in a circulating flow of informa¬ 
tion, before it can be broadcast globally and become 
conscious (e.g. Scheibel and Scheibel, 1965; Shevrin 
and Dickman, 1980). Dixon (1971) has also argued that 
a circulating flow of information between the reticu¬ 
lar formation and the sensory areas of the cortex is 
required before sensory input becomes conscious. 

The possible role of the ERTAS in conscious experi¬ 
ence is an intriguing one! It makes intuitive sense that 
there must be some kind of broadcast system in the 
brain that allows for all modes of sensory processing - 
sight, hearing, touch - to combine with conscious 
thought and experience in order to focus on some 
inputs and suppress others. Clearly, the ERTAS does 
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not work in isolation in these types of brain functions. 
The thalami and regions in the prefrontal cortex are 
likely closely intertwined with ERTAS-like processes. 
Nevertheless, ERTAS seems to play a key role in 
human conscious experience. 

4*0 SUMMARY 

It is a difficult task indeed to attempt to describe a 
dynamic and complex biological structure such as 
the brain in a few short pages. Our aim in this chap¬ 
ter was to provide you with the basic structures and 
regions of the brain and their function in human cogni¬ 
tion. Some important points to remember are that the 
brain has developed and changed through time and 
so some areas of the brain are 'older' than others. The 
cortex or neocortex represents recent brain develop¬ 
ments in the human, and the frontal and parietal lobes 
have expanded their neural territory tremendously as 


compared to non-human primates. While there are sep¬ 
arable regions and parts of the brain, such as the two 
hemispheres and the four major lobes, nonetheless, the 
brain is highly interconnected with an extensive fiber 
pathway system that connects the hemispheres, the 
lobes, and provides circuits to subcortical regions. 

Some important questions about human brain 
structure and function remain a puzzle to us. Why do 
we see so much evidence of duality in the brain, with 
two hemispheres, two thalami, for example, when we 
have one mind? What role do the mirror image regions 
of the brain play in human cognition? While some 
'newer' regions of the brain, such as the prefrontal cor¬ 
tex and the inferior parietal lobe, seem to be the site 
for higher order associative cognition, there are also 
some ancient regions, such as the reticular formation, 
that seem to play a key role in consciousness. New and 
ancient, the many regions of the brain come together to 
form a dynamic and intricate biological structure that 
holds many more puzzles for scientists to unravel. 


5*0 CHAPTER REVIEW 
5 ♦ 1 Study questions 

1 Why is cortex sometimes referred to as 
'neocortex'? 

2 What are the four major lobes of the brain and 
what are some of their key functions in human 
cognition? 

3 Where is the medial temporal lobe located? 
What are its key structures? 


4 Briefly describe the role of the thalami in brain 
processing. 

5 How are the hemispheres linked? Are there any 
differences in how they function? 

6 What is the reticular formation and what role 
may it play in conscious thought? 

5*2 Drawing exercises 

Show the locations and names of the major brain 

landmarks using Figure 5.41 on page 156. 
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The question is not what you look at, but what you see. 


Henry David Thoreau 



An abstract figure showing a rotating checkerboard stimulus, which highly 
activates the early visual areas located in the occipital lobe (in blue). Notice that 
we are looking at a medial view of the right hemisphere. Source : Buckley and 
Gaffan, 2006. 
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1.0 INTRODUCTION 

1.1 The mystery of visual experience 

Think back to the last time you stood aloft a high look¬ 
out, looking down at the world below. Perhaps you 
were at the top of a summit and could see the wide 


expanse of the horizon cut by repeating mountain- 
tops, forests, valleys, and rivers below, or perhaps you 
were at the top of a tall building as the bustling swirl¬ 
ing colors of rush-hour traffic, cars, and people flowed 
through the streets. It's worth pausing for a moment to 
think about how such vivid and powerful impressions 
can be the simple result of a collection of neurons firing 
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inside your brain. How can the collective electrical 
activity of billions of neurons be responsible for 
everything you are seeing and experiencing at this 
very moment? These are some of the perplexing but 
fascinating questions that arise when we consider the 
relationship between brain activity and subjective 
visual experience. 

Most people intuitively think that human vision 
works much like a camera (Figure 6.1(a)). As we go 
about our daily activities, it is easy to believe that we 
see our entire world in crisp full-resolution color - much 
like a high-resolution photograph. However, as it turns 
out, this is far from the case. Due partly to the structure 
of our eyes (which we will revisit shortly), our visual 
perception is in full color and high resolution only at the 
center of gaze. This may seem hard to believe, as you 
dart your eyes around the room now; the illusion that 
you are experiencing the whole visual scene in full color 
and clarity seems convincing. However, if you fix your 
gaze at a particular point in space - say the corner of the 
windowsill or this word on the page - and get someone 
to hold up some fingers out at the edge of your field of 
view, you will find that without moving your eyes it is 
extremely hard to count the number of fingers he or she 
holds up. Also, you may notice that your experience of 
color is somehow dull and lacking in richness. 

There are several reasons why it makes sense to 
restrict high-resolution vision to only a small portion of 
our visual space - this is just one of the many strategies 


that the brain uses to help represent the specific fea¬ 
tures and objects we see in the most efficient and 
effective way possible. It is a testament to the brain's 
ability seamlessly to represent the outside world that 
we normally remain oblivious to this striking fact. 


1.2 The purpose of vision: knowing what is 
where 

For most people, vision may be the most important of 
the five senses for getting around in everyday life and 
getting things done. What is the purpose of vision? 
David Marr, an early vision scientist and computer 
expert, made the deceptively simple comment that the 
goal of vision is 'to know what is where'. For example, if 
you are walking or driving to the university to find a 
class in a new building, it is important to know: where 
are the other cars and pedestrians , is the light red or green , 
which way is that car going , how fast is it approaching , do I 
turn at this street corner, is this the building I'm looking for? 

Considering the goal of vision, it becomes clear that 
visual perception is far more complicated than simply 
taking a picture with a digital camera (Figure 6.1(a)). A 
digital camera can capture the image projected from the 
environment and store it in an array of pixels. However, 
the camera doesn't really do anything with this image 
and doesn't have any knowledge about what is stored in 
the image, such as what objects are in the photo or where 



FIGURE 6.1 Visual experiences, (a) Just one of millions of images you can experience with your visual system. 
Unlike the camera, you actually experience the image; you know what it is you are looking at. (Image courtesy of 
the Tong lab.) (b) Another example of the detailed, multifeature capabilities of your visual system. You can differen¬ 
tiate many different orientations, colors, and shapes. Source: Frank Tong, with permission. 
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they are. The easiest way to figure out what is in the pic¬ 
ture is to have someone look at the picture and interpret 
it with his or her brain. Visual perception is what hap¬ 
pens after the picture reaches the eyes - the image forms 
a pattern of activity on the array of receptors in the eye, 
and the detailed pattern is analyzed by the visual centers 
of the brain, thereby revealing what is where. 


and so forth (Figure 6.2). These laws suggest that ele¬ 
ments more similar in color or shape are more likely to 
be perceived as a group. Likewise, if a set of elements 
is arranged in a way that they are more closely spaced 
in rows or columns, this will determine whether they 
are perceived as rows or columns. 

Why is perceptual grouping so important? It helps 
us perceive which features belong to a possible object. 


1.3 Knowing what: perceiving features, 
groups, and objects 

How does the brain perceive what something is? 
Studies of human visual perception and neuroscience 
suggest that there are many levels of perception. At the 
most basic level, the human brain appears to process 
basic visual features such as color, orientation, motion, 
texture , and stereoscopic depth. For example, when look¬ 
ing at the picture of the flower shown in Figure 6.1(b), 
we may perceive that the center of the flower is yellow, 
the leaf just below is green, and the two stems are each 
at different angles. We are very good at perceiving small 
differences in orientation (1-2 degrees of angular tilt), 
subtle differences in color (e.g. the red of a rose or the 
red of a strawberry), and very faint traces of motion. 

As we will see in the next section, most neurons in 
early visual areas of the brain are highly tuned to spe¬ 
cific features - some may fire very strongly to a line 
shown at a particular angle, to a particular color, or to 
a particular motion direction. These neurons respond 
to a very small region of the visual field (i.e. your cur¬ 
rent field of view), ranging from just a fraction of a 
degree to a few degrees of visual angle. (If you hold 
your thumb at arm's length, the width of your thumb 
is probably about two degrees of visual angle (O'Shea, 
1991). (The moon, when viewed from Earth, is about 
0.5 degrees in diameter.) 

If the activity of each of these neurons represents 
only a small part of the visual field, such as whether a 
small patch of the visual field contains vertical or hori¬ 
zontal, red or blue, motion or something stationary, then 
how is the brain able to combine this information across 
many neurons? Somehow, the brain is able to organize 
these basic feature elements into organized perceptual 
groups. The Gestalt psychologists proposed that percep¬ 
tion could not be understood by simply studying the 
basic elements of perception (Wertheimer, 1912; Koffka, 
1935). The German word. Gestalt , is difficult to translate 
directly, but expresses the idea that the whole is greater 
than the sum of the parts. These psychologists proposed 
the Gestalt laws of perceptual grouping , such as the laws 
of similarity, proximity, good continuation, common fate, 



(a) 


Grouping by proximity 



(b) 


Grouping by good continuation 



FIGURE 6.2 Gestalt grouping, (a) Grouping by similarity, the 
white dots are grouped with other white dots. On the right, the squares 
group with squares. The objects with similar features tend to group 
together, (b) Here, even though all the objects are circles, due to their 
grouped locations or proximity to each other, we perceive two sepa¬ 
rate groups of dots, (c) Grouping by good continuation. On the left, 
we perceive one object. On the right, the exact same lines are drawn 
but separated so that there is no continuation across the entire object. 
In all these cases, the collections of objects form groups or larger glo¬ 
bal objects, which are greater than the simple sum of the parts. Source: 
Frank Tong, with permission. 
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and helps us distinguish an object from the back¬ 
ground. For example, imagine seeing a golden retriever 
lying in tall grass. Grouping by similarity may help 
us see that the dog is separate from the grass, and the 
neighboring wagging tail might be recognized as part 
of the dog, even though only the end of the tail can be 
seen. If the dog starts to run toward a bush, barking 
furiously, the cat hidden behind the bush may be invis¬ 
ible until it starts to move, because the cat's motion 
leads to grouping by common fate, which may evoke 
a vivid impression of the complete shape of the ani¬ 
mal rather than bits and fragments of fur separated by 
branches and leaves. 

Finally, we can perceive the shape of entire objects , 
and match these shape representations to the objects 
we know from previous experience. To perceive an 
object, the brain has to go through many stages of vis¬ 
ual processing, from processing the feature elements of 
the object, putting the elements together into coherent 
groups, and finally figuring out how those elements 
form a coherent organized shape. This representation of 
the object's shape must then be matched to the correct 
object representation stored in memory. Given that there 
are thousands of objects and that the two-dimensional 
image of any object projected onto the back of the eye 
can vary from changes in viewpoint, lighting, or view¬ 
ing conditions, this makes the problem of object recog¬ 
nition especially challenging. Somehow, the brain must 
abstract the stable or invariant properties of an object 
while disregarding all the superficial ways in which the 
2D image of an object can vary. 

Later in this chapter, we will discuss how the brain 
processes different types of visual features and objects. 

1.4 Knowing where things are 

How do we know where objects are located in the world? 
When we look at the world, the image that strikes the 
back of our eye is essentially two-dimensional, similar to 
the image that would be taken by a camera. This two- 
dimensional map of the world projected onto the eye is 
preserved in the early visual areas of the cerebral cortex, 
which provides a map of where objects are located rela¬ 
tive to the center of gaze. The brain is also able to figure 
out the missing third dimension and estimate how far 
away objects are in space. Whereas early visual areas 
represent the positions of objects relative to the center 
of gaze, higher brain areas in the parietal, temporal, or 
frontal lobe are more likely to represent the position of 
objects in a more abstract (less visual) manner, relative 
to the person's body position or relative to the global 
environment. 


2.0 FUNCTIONAL ORGANIZATION OF 
THE VISUAL SYSTEM 

When the light coming from an object reaches our eyes, 
it triggers a cascade of neural events as this visual pat¬ 
tern is converted into neural impulses that travel up the 
visual system, from one brain area to the next. Through 
a series of neural processes in many brain areas, the 
activity of neurons in numerous brain areas somehow 
leads to the visual experience and recognition of the 
object and its many component features. Let's trace the 
sequence of events to understand how the brain pro¬ 
cesses visual information at each stage of visual process¬ 
ing. This will help us understand how different visual 
areas of the brain contribute to visual perception. 

2.1 The retina 

There are two types of photoreceptors: cones and rods 
(Figure 6.3). Cones are color-selective, less sensitive to 
dim light than rods, and important for detailed color 
vision in daylight. Each cone contains one of three kinds 
of photopigments , specialized proteins that are sensitive 
to different wavelengths of light. These wavelengths 
roughly correspond to our ability to distinguish red, 
green, and blue. When light strikes a photopigment 
molecule, the light energy is absorbed and the molecule 
then changes shape in a way that modifies the flow of 
electrical current in that photoreceptor neuron. Cones 
are densely packed into the fovea, the central part of the 
retina that we use to look directly at objects to perceive 
their fine details. In the periphery, cones are more spread 
out and scattered, which is why objects in the periphery 
appear blurrier and their colors are less vivid. 

Rods contain a different photopigment that is much 
more sensitive to low levels of light. Rods are impor¬ 
tant for night vision - we rely on seeing with our rods 
once our eyes have adapted to the darkness (dark 
adaptation). Curiously, there are no rods in the fovea, 
only cones, and the proportion of rods increases in the 
periphery. This is why you may have noticed when 
gazing at the night sky that a very faint star may be 
easier to see if you look slightly off to one side. 

The signals from photoreceptors are processed by a 
collection of intermediary neurons, bipolar cells , horizon¬ 
tal cells , and amacrine cells , before they reach the ganglion 
cells , the final processing stage in the retina before sig¬ 
nals leave the eye. The actual cell bodies of ganglion 
cells are located in the retina, but these cells have long 
axons that leave the retina at the blind spot and form 
the optic nerve. Each ganglion cell receives excitatory 
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FIGURE 63 The eye. (a) Illustration showing how objects in the environment are physically pro¬ 
jected to the back of the eye - the retina, (b) The eye and a cross-section of the retina. The cross-section 
of the eye shows where the photoreceptors are located in the retina. Both the rods and cones are shown. 
They respond to different types of light. The neural signal then travels via bipolar cells and then to the 
ganglion cells. The axons of the ganglion cells take the neural information out of the eye and backward 
toward the cortex. Source : Squire et al., 2003. 


inputs from a collection of rods and cones - this distilla¬ 
tion of information forms a receptive field - a concept we 
will revisit shortly. Ganglion cells at the fovea receive 
information from only a small number of cones, while 
ganglion cells in the periphery receive inputs from 
many rods (sometimes thousands). With so many rods 
providing converging input to a single ganglion cell, if 
any one of these rods is activated by photons of light, 
this may activate the cell, which increases the likelihood 
of being able to detect dim scattered light. However, 
this increase in sensitivity to dim light is achieved at the 
cost of poorer resolution - rods provide more sensitiv¬ 
ity, but also a 'blunder' picture than the sharp daytime 
image provided by cone vision. 

Retinal ganglion cells receive both excitatory and 
inhibitory inputs from bipolar neurons, and the spatial 
pattern of these inputs determines the cell's receptive 
field (Figure 6.4(a)). A neuron's receptive field refers 
to the portion of the visual field that can activate or 
strongly inhibit the response of that cell. Retinal gan¬ 
glion neurons have center-surround receptive fields. 
For example, a cell with an on-center off-surround recep¬ 
tive field will respond strongly if a spot of light is pre¬ 
sented at the center of the receptive field. As that spot of 
light is enlarged, responses will increase up to the point 


where light begins to spread beyond the boundaries 
of the on-center region. After that, the response of the 
ganglion cell starts to decline as the spot of light gets 
bigger and stimulates more and more of the surround¬ 
ing off-region. (A cell with an off-center on-surround 
receptive field will respond best to a dark spot pre¬ 
sented in the center of the receptive field.) 

How can the behavior of retinal ganglion cells be 
understood? A key concept is that of lateral inhibition 
(Kuffler, 1953). Lateral inhibition means that the activ¬ 
ity of a neuron may be inhibited by inputs coming from 
neurons that respond to neighboring regions of the visual 
field. For example, the retinal ganglion cell in Figure 6.4(b) 
receives excitatory inputs from cells corresponding to the 
on-center region, and inhibitory inputs from the off-center 
region. The strength of these excitatory and inhibitory 
inputs are usually balanced, so that if uniform light is pre¬ 
sented across both on- and off-regions, the neuron will not 
respond to uniform illumination. 

Why are center-surround receptive fields and lateral 
inhibition so important? Lateral inhibition is impor¬ 
tant for enhancing the neural representation of edges , 
regions of an image where the light intensity sharply 
changes. These sudden changes indicate the presence 
of possible contours, features, shapes, or objects in any 
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FIGURE 6.4 Center-surround receptive fields, (a) Schematic example of a center-surround cell's 
response to different-sized patches of light. Notice that the biggest spiking response (shown by the lines on 
the right) occurs for the intermediate-sized center light patch. The spot of light has to be just the right size 
to get the maximum response out of that particular neuron, (b) A model of how a center-surround receptive 
field might be achieved by the collaboration and competition between different connective neurons in the 
retina. Source : Frank Tong, with permission. 
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FIGURE 6.5 The edges hold most information. An example of how most of the information in the picture 
comes from the edges of objects. Figure on the left is the original, on the right is information from the edges only - 
taken from the image using a computer algorithm. Source : Frank Tong, with permission. 


visual scene, whereas uniform parts of a picture are not 
particularly informative or interesting. Figure 6.5 shows 
a picture of the fox in original form and after using 
a computer to filter out just the edges (right picture), 
so that the regions in black show where ganglion cells 
would respond most strongly to the image. Lateral inhi¬ 
bition also leads to more efficient neural representation, 
because only the neurons corresponding to the edge of a 
stimulus will fire strongly; other neurons with receptive 
fields that lie in a uniform region do not. Because the 
firing of neurons takes of a lot of metabolic energy, this is 
much more efficient. This is an example of efficient neural 


coding - only a small number of neurons need to be active 
at any time to represent a particular visual stimulus. 

Lateral inhibition also helps ensure that the brain 
responds in a similar way to an object or a visual scene 
on a dim gray day and on a bright day. Changes in the 
absolute level of brightness won't affect the pattern of 
activity on the retina very much at all; it is the relative 
brightness of objects that matters most. Finally, lat¬ 
eral inhibition at multiple levels of visual processing, 
including the retina, lateral geniculate nucleus (LGN), 
and visual cortex, may lead to interesting visual illu¬ 
sions such as the Hermann grid illusion (Figure 6.6). 
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FIGURE 6.6 Hermann grid illusion. Take a careful look at the 
collection of black squares in the figure. Do you notice anything 
unusual? Do you have the impression of seeing small dark circles in 
between the black squares in the periphery? Don't be alarmed, this 
is completely normal. This is a great example of receptive fields with 
lateral inhibition at work (Herman, 1870). In the rightmost matrix of 
squares, some possible receptive fields are shown. A receptive field 
that falls between the corners of four dark squares will have more 
of its inhibitory surround stimulated by the white parts of the grid 
than a receptive field that lies between just two of the dark squares. 
As a result, neurons with receptive fields positioned between four 
dark squares will fire more weakly, leading to the impression of 
small dark patches at these cross points. At the fovea, receptive 
fields are much smaller so the illusion is only seen in the periphery 
Source: Frank Tong, with permission. 


2.2 Lateral geniculate nucleus (LGN) 

From the eye, retinal ganglion cells send their axons 
to a structure in the thalamus called the lateral genic¬ 
ulate nucleus (LGN). Specifically, the left half of each 
retina projects to the left LGN; the right half of the 
retina projects to the right LGN. For this to happen, 
the inputs from the nasal portion of each retina must 
cross at the optic chiasm to project to the opposite 
LGN (Figure 6.7). The result is that the left LGN 
receives input from the right visual field, and the right 
LGN receives input from the left visual field, so that 
each LGN serves to represent the contralateral (i.e. 
opposite) visual field. Note that the inputs from each 
eye go to separate monocular layers of the LGN, so 
signals from the two eyes remain separate until they 
reach the primary visual cortex where these signals are 
combined. 

What do the receptive fields in the LGN look like? 
Well, they share the same shape and basic properties of 
the retinal ganglion cells with center-surround receptive 
fields. The thalamus is often considered a way station 
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FIGURE 6.7 The visual pathways from retina to cortex, (a) Example of a brain slice from a functional 
magnetic resonance imaging (fMRI) scan, showing the lateral geniculate nucleus (LGN) and primary vis¬ 
ual areas at the back of the brain (the occipital cortex). The two different colors denote the two hemispheres 
of the brain, (b) Schematic illustration showing the visual pathways from the retina in the eyes to the pri¬ 
mary visual cortex at the back of the brain. You can see here that the neural information from the nasal 
or inner sides of the eyes crosses over at the optic chiasm, to be processed in the contralateral side of the 
brain. The left visual field, in blue, is processed by the right visual cortex (also blue). The LGN, displayed 
in green, relays the visual information to the primary visual areas of the cortex. Source : Squire et al., 2003. 
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FIGURE 6.8 Orientation selectivity 
in VI. An example of how a collection of 
center-surround receptive fields could lead 
to orientation selectivity in VI neurons. The 
overlapping circles on the left show center- 
surround receptive fields. When the bar of 
light lays vertically it triggers all the on- 
centers (+) of each receptive field, whereas 
when its orientation changes (the bar tilts) 
fewer centers and more surrounds are acti¬ 
vated resulting in a smaller neural response 
(fewer spike bars in the graph). Hence the 
vertical bar gives a larger neural response, 
when the stimulus is oriented the magni¬ 
tude of the VI response is reduced. This 
constitutes orientation selectivity - only 
observed in the cortex. Source : Frank Tong, 
with permission. 


for signals finally to reach the cerebral cortex, where the 
neurons start to respond in very different ways. 

So how would LGN neurons or retinal ganglion cells 
respond to the simple outline of the house, as is illus¬ 
trated in Figure 6.10? Cells with on-center receptive fields 
positioned on any of the contours of the house would 
fire strongly. Note, though, that these cells will fire just as 
strongly for a vertical, horizontal, or tilted line. Although 
a small portion of the inhibitory surround is stimulated 
by these lines, the entire proportion of the on-center 
region is being stimulated, leading to greater excitation 
than inhibition, and therefore a steady level of firing. 

23 Primary - or striate - visual cortex (V1) 

From the LGN, neurons send their signals to the primary 
visual cortex, sometimes called VI because this region is 
the first cortical visual area. VI is frequently referred to as 
'striate' cortex because of its distinguishing striped - or 
striate - appearance. About 90 percent of the outputs 
from the retina project first to the LGN and then onward 
to VI. The left LGN projects to VI in the left hemisphere; 
the right LGN projects to right VI (Figure 6.7). In VI, the 
spatial layout of inputs from the retina is still preserved. 
Left VI receives an orderly set of inputs from the left 
half of both retinas, via the thalamus. The foveal part 
of the visual field is represented in the posterior part of 
the occipital lobe, near the occipital pole and the more 
peripheral parts of the visual field are represented more 
anteriorly. Left VI therefore contains a retinotopic map 
of the entire right visual field, while right VI contains a 
map of the entire left visual field. This retinotopic organi¬ 
zation is very prevalent in early visual areas (VI through 
V4), where neurons have small receptive fields, but 
becomes weaker and less orderly in higher visual areas 
outside of the occipital lobe. 


Neurons in VI are sensitive to a whole host of vis¬ 
ual features, not seen in the LGN. One of the most 
important visual features is orientation (Hubei and 
Wiesel, 1962, 1968). Some VI neurons respond best to 
vertical lines, some to 20-degree tilted lines, others to 
horizontal lines and so forth. How do these neurons 
attain these new properties or receptive fields? Figure 
6.8 shows an example of a model for VI orientation 
selectivity. If a VI neuron receives excitatory input 
from three LGN neurons with aligned center-surround 
receptive fields, then the VI neuron will respond best 
to a matching oriented line. For example, if a vertical 
bar is presented, the neuron shown in the figure will 
respond at its strongest, because the entire excita¬ 
tory region will be stimulated, whereas the inhibitory 
surround will not be stimulated. If the bar is tilted 
somewhat away from vertical, the neuron will 
respond more weakly because part of the inhibitory 
surround will now be stimulated and part of the exci¬ 
tatory center will not. Finally, if a horizontal bar is pre¬ 
sented, the neuron may not fire at all, because equal 
proportions of the center and the surround regions 
will be receiving stimulation, leading to a balance in 
the strength of incoming excitatory and inhibitory 
inputs. This configuration of center-surround recep¬ 
tive fields can explain the orientation selectivity of VI 
neurons. 

VI neurons are also sensitive to many other visual 
features besides orientation (Hubei and Wiesel, 1998). 
Some neurons respond best to a particular direction 
of motion, such as upward motion, leftward motion, 
or downward motion. Other neurons respond best to 
particular colors or color differences (e.g. red versus 
green, yellow versus blue), though some basic types 
of color-sensitive neurons can also be found in the 
retina and LGN. Finally, some neurons respond best 
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FIGURE 6.9 Pathways in the visual 
system: from the eye to VI. A schematic 
drawing of the pathways in binocular 
vision, showing the visual input from 
left and right visual fields (top of figure) 
through the optic nerve and optic tract 
(center of figure), continuing on through 
the LGN and onto VI in cortex (bottom 
of the figure). Source : Standring, 2005. 


to particular binocular disparities (Barlow et ad., 1967; 
Cumming, 2002), which refer to the degree of align¬ 
ment between images in the two eyes. Small displace¬ 
ments between images in the two eyes are what allow 
us to perceive stereo-depth when we look at objects 
with both eyes open. (Try closing one eye, extending 
your arms to full length and try quickly to move your 
two index fingers to meet one another. Now try this 
again with both eyes open. If you have normal binoc¬ 
ular vision, this should be much easier to do with both 
eyes open because you can better judge the distance of 
your two fingers.) See Figure 6.9 for a schematic view 
of pathways in the binocular vision system. 


So how will VI neurons respond to the outline of the 
house? A VI neuron that is tuned to 45-degree tilted 
lines and has its receptive field in the position along the 
roof may fire strongly to the angled roof. A VI neuron 
that responds best to vertical will help signal the pres¬ 
ence of the vertical wall and a horizontal neuron will 
respond to the ceiling or floor of the house. In this sense, 
it can be readily seen that VI neurons do much more 
than respond to simple spots of light, as the LGN does. 
VI neurons provide a neural representation of the orien¬ 
tation of visual features that comprise the contours and 
shapes of objects. Figure 6.10 provides a summary of 
the hierarchy of visual processing. From the LGN, VI, 
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FIGURE 6.10 The hierarchy of visual processing. A demonstration of the hierarchical response prop¬ 
erties of the visual system to simple and complex stimuli. The leftmost column shows our house stimulus 
and what receptive fields of each visual area we would see in the balloons. Not only do the receptive field 
sizes increase in each visual area, but also the complexity of the shapes they respond to. The rightmost 
column shows an estimate of where each area is in the brain. You can see that early visual areas respond 
to simple features and, as we move along the processing stream, areas respond to more complex shapes 
and objects. This is a well-established theme of the visual system. Source : Frank Tong, with permission. 


V4 to the ventral temporal cortex, you can see that 
neurons gradually respond to more complex stimuli 
from one area to the next. 

So, to summarize, VI is important in analyzing the 
visual features at a fine level of detail. These neurons 
have small receptive fields that are sensitive to orien¬ 
tation, color, motion, or binocular disparity. After vis¬ 
ual signals are analyzed in VI, they are sent to higher 
visual areas for further processing. 

2.4 Extrastriate visual areas - outside of VI 

VI sends feedforward signals to many higher visual areas, 
including areas such as V2, V3, V4, and motion-sensitive 
area MT, to name a few (Figure 6.11) (Felleman and Van 
Essen, 1991). Area V4 is known to be especially important 
for the perception of color (Zeki, 1977) and some neurons in 
this area respond well to more complex features or com¬ 
binations of features (Pasupathy and Connor, 2002). For 
example, some V4 neurons are sensitive to curvature or 
to two lines that meet at a specific angle. These neurons 
might signal the presence of a curving contour or a corner. 


From our example of the house, a V4 neuron might 
respond best to the meeting of the two lines forming the 
point of the roof or to another comer of the house. 

How then are these various bits and parts of the 
house, as represented by simple line orientations and 
corners, eventually represented as an entire object? 
Area V4 sends many outputs to higher visual areas 
in the ventral visual pathway, which is important for 
object recognition (Ungerleider and Mishkin, 1982). 
The anterior part of the ventral visual pathway con¬ 
sists of the ventral temporal cortex, which is especially 
important for object recognition. 

2.5 Area MT 

The middle-temporal area, or what is commonly called 
area MT (see Figure 6.11), is important for motion per¬ 
ception. Almost all of the neurons in area MT are direc¬ 
tion-selective, meaning that they respond selectively to 
a certain range of motion directions and do not respond 
to directions beyond that range (Zeki, 1974; Albright, 
1984). Moreover, some of these neurons respond well to 
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FIGURE 6.11 Visual areas of the brain. Functionally defined visual areas of the 
human brain, shown on a reconstruction from anatomical brain scans, (a) A flattened 
cortical representation of the human visual system. The flat map was created by unfold¬ 
ing the convoluted surface of the cerebral cortex, applying a cut through the calcarine 
sulcus, which divides area VI into dorsal and ventral halves, and flattening the corti¬ 
cal sheet using computerized methods. This allows one to see all the visual areas of one 
hemisphere, including those buried in the sulci, in a single image. The upper half shows 
dorsal portions of visual areas VI, V2, and V3, as well as V3A. The lower half shows 
ventral visual areas VI, V2, V3, and V4, as well as the parahippocampal place area (PPA) 
and fusiform face area (FFA). Areas LOC and MT are also shown, (b) A ventral view of 
one hemisphere, showing the location of these areas in the cortex, (c) Lateral view of the 
same hemisphere, showing the positions of the different visual areas, (d) A medial view 
of the hemisphere showing mainly VI. Source: Squire et ah, 2003. 


patterns of motion (Albright, 1992), meaning that 
these neurons can integrate many different motion 
directions and calculate what the overall direction of 
an object might be. As we will see, the activity in this 
region seems to be closely related to motion percep¬ 
tion and, when activity in this region is disrupted, 
motion perception may be severely impaired. 

2.6 The ventral and dorsal pathways: 
knowing what and where 

The projections from VI to higher areas in the cortex 
can be roughly divided according to two major paral¬ 
lel pathways: a ventral pathway leading from VI to the 
temporal lobe that is important for representing 'what' 
objects are and a dorsal pathway leading from VI to the 
parietal lobe that is important for representing 'where' 
objects are located (Figure 6.12). 

This distinction between the ventral and dorsal 
pathways, sometimes referred to as the what and where 



FIGURE 6.12 What and where pathways. The 'where' path¬ 
way is typically called the dorsal pathway because it includes dor¬ 
sal areas like MT and the parietal cortex that are along the top of the 
brain. The 'what' pathway includes ventral areas like V4, LOC, and 
IT; hence it is known as the ventral processing pathway These two 
pathways form a nice way of visualizing the flow of different corti¬ 
cal information. Source: Squire et ah, 2003. 
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FIGURE 6.13 Dorsal and ven¬ 
tral visual pathways. From the pio¬ 
neering work of Ungerleider and 
Mishkin (1982), a schematic drawing 
of the dorsal and ventral pathways 
based on studies in monkey, (a) A 
lateral view of monkey brain, show¬ 
ing the location of primary vision 
cortex (VI), extra-striate cortex (V2- 
V4), and other key vision areas such 
as MT. (b) A schematic showing 
visual areas involved in dorsal and 
ventral processing streams. Source : 
Squire et al., 2003. 


pathways, respectively, is an important organiza¬ 
tional principle of the visual system proposed by 
Ungerleider and Mishkin (1982) (Figure 6.13). 

Support for this distinction comes from what has 
been discovered about the anatomical connections 
between visual areas in these two pathways, the 
response properties of visual areas in each pathway, 
and the visual deficits that can result from damage to 
the parietal or temporal lobe. 

In the dorsal pathway, signals from VI travel to dor¬ 
sal extrastriate areas, such as area MT and V3A, which 
then send major projections to many regions of the pari¬ 
etal lobe. The dorsal pathway is important for repre¬ 
senting the locations of objects, so that the visual system 
can guide actions toward those objects (Goodale and 
Humphrey, 1998). Consider what is involved in reach¬ 
ing for any object, such as a coffee mug sitting on a 
table; this type of vision requires detailed information 
about the precise location, size, and orientation of the 
object. Without such detailed information, you might 


reach toward the wrong location, your grasp might not 
match the size of the handle of the mug, or your hand 
might not be angled properly for gripping the handle. 
Areas MT and V3A are important for processing visual 
motion and stereo-depth, while specific regions in the 
parietal lobe are specialized for guiding eye move¬ 
ments or hand movements to specific locations in vis¬ 
ual space. 

In the ventral pathway, many signals from VI travel 
to ventral extrastriate areas V2, V3, and V4 and onward 
to many areas of the temporal lobe. The ventral or 
'what' pathway is important for processing information 
about the color, shape, and identity of visual objects, 
processing which emphasizes the stable, invariant prop¬ 
erties of objects. For example, the ventral pathway is less 
concerned about the exact size, orientation, and position 
of the coffee mug; instead, its goal is to be able to iden¬ 
tify such an object anywhere in the visual field and to be 
able to tell it apart from other similar objects (e.g. cups, 
bowls, teapots). We will be talking about the properties 
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of the ventral pathway in much greater detail, espe¬ 
cially different regions of the temporal lobe. 

While this dorsal-ventral pathway distinction is 
useful for grouping areas of the brain and under¬ 
standing how much of the information flows back and 
forth between visual areas, it should not be taken as 
an absolute distinction. There is plenty of cross talk 
between the two pathways. Also, the parietal and tem¬ 
poral lobes send projections to some common regions 
in the prefrontal cortex, where information from each 
pathway can also be reunited. 

2.7 Areas involved in object recognition 

Single-unit recordings in monkeys have revealed that 
many neurons in the ventral temporal cortex respond 
best to contours, simple shapes, or complex objects. 
For example, some neurons in this region may respond 
best to square or round shapes, others to triangles and 
others still to even more complex figures, such as the 
outline of a house (see Figure 6.10), which looks like 
a triangle lying on top of a square. Some neurons in 
this region are highly selective and respond to only a 
particular kind of object, such as a hand, a face shown 
from a particular viewpoint, a particular animal, a 
familiar toy, or an object that the monkey has learned 
to recognize and so forth (Desimone et al, 1984; Gross, 
1992; Logothetis et al., 1995; Tanaka, 1996). 

Human neuroimaging studies have revealed many 
brain areas involved in processing objects. These 
object-sensitive areas, which lie just anterior to early 
visual areas V1-V4, respond more strongly to coherent 
shapes and objects, as compared to scrambled, mean¬ 
ingless stimuli. In this chapter, we will focus on three 
such brain areas (see Figure 6.10). The lateral occipital 
complex (LOC) lies on the lateral surface of the occipi¬ 
tal lobe, just posterior to area MT. Because this region 
is strongly involved in object recognition, we will con¬ 
sider it as part of the ventral pathway, even though its 
position is quite dorsal when compared to other object 
areas. The fusiform face area lies on the fusiform gyrus, 
on the ventral surface of the posterior temporal lobe. 
The parahippocampal place area lies on the parahippo¬ 
campal gyms, which lies just medial to the fusiform 
gyrus on the ventral surface of the temporal lobe. 

2*8 Lateral occipital complex (LOC) 

The lateral occipital complex seems to have a general 
role in object recognition and responds strongly to a 
variety of shapes and objects (Malach et al., 1995). 
Figure 6.14 shows an example of the neural activity 
in LOC compared to VI. As the picture becomes more 


and more scrambled, VI continues to respond and 
even gives a larger response, whereas activity in LOC 
declines. This shows that LOC prefers intact shapes 
and objects more than scrambled visual features. 

This region seems to represent the particular shapes 
of objects. Presumably, different neurons in this region 
respond best to different kinds of objects. Because human 
brain imaging lacks the resolution to measure the object 
preferences of individual neurons, another method to 
test for object selectivity is to measure neural adaptation to 
a particular shape. This involves presenting two objects 
in a row - if the same object shape is presented twice in 
a row, then a region with object-selective neurons should 
adapt to the repeated shape and respond more weakly 
the second time around. This is exactly what LOC does, 
even when the repeated object is presented in a new 
location or in a different format so that retinotopic visual 
areas (V1-V4) won't adapt to the image (Grill-Spector 
et al., 1999; Kourtzi and Kanwisher, 2001). 

2*9 Fusiform face area (FFA) 

Neurophysiological studies have shown that face- 
selective neurons can be found in many parts of the 
temporal lobe of monkeys. Some of these face cells show 
remarkable precision in what they respond to and might 
respond best to a face of a particular identity, facial 
expression, or to a particular viewpoint of a face (e.g. right 
profile). Usually, these face cells can be found intermixed 
with neurons that respond to different types of objects, 
which led scientists to believe that there might not be a 
part of the brain that is dedicated to processing faces. 

However, human neuroimaging studies have shown 
that there is a region in the fusiform gyms, called the 
fusiform face area (FFA), that responds more strongly to 
faces than to just about any other category of objects 
(Kanwisher et al, 1997). This region responds more to 
human, animal, and cartoon faces than to a variety of 
non-face stimuli, including hands, bodies, eyes shown 
alone, back views of heads, flowers, buildings, and 
inanimate objects (Kanwisher et al., 1997; McCarthy 
et al, 1997; Tong et al, 2000; Schwarzlose et al., 2005). In 
a recent study, researchers tried scanning the brains of 
monkeys to see if they might also have a face-selective 
area in the ventral temporal cortex and it turns out that 
they do too (Tsao et al, 2006). The researchers could 
then record the activity of single neurons in this face 
area and discovered that 97 percent of the neurons in 
this region responded more to faces than to other kinds 
of objects. Moreover, these neurons were very good at 
telling apart different identities of faces but poor at tell¬ 
ing apart different identities of objects, suggesting they 
may have an important role in recognizing and telling 
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FIGURE 6.14 Neural response from low and high level areas. The response of primary visual cortex (VI) and 
lateral occipital (LOC) to a picture of a kitten at different coherencies. As the picture is scrambled VI continues to 
respond - in fact, it actually increases its response once the image is scrambled. This demonstrates that it is not the 
image of the kitten which is driving responses in VI, but local patches of luminance and orientation. Conversely, the 
activity in lateral occipital cortex shows a large response to the kitten but, as the picture is scrambled, the activity 
in LOC drops down dramatically. This demonstrates that unlike VI, the activity in LOC is in response to the kitten. 
Source : Frank Tong, with permission. 


apart different faces. Because this region is very small, 
this highly face-selective region that consists almost 
entirely of face cells was never discovered before. As 
we will see, this region seems to be important for the 
conscious perception of faces. 

2.10 Parahippocampal place area (PPA) 

The parahippocampal place area (PPA) is another 
strongly category-selective region that responds best to 
houses, landmarks, indoor and outdoor scenes (Epstein 
and Kanwisher, 1998). In comparison, this brain area 
responds more weakly to other types of stimuli, such as 
faces, bodies or inanimate objects. Because this region 


responds to very different stimuli than the fusiform face 
area, many studies have taken advantage of the differ¬ 
ent response properties of the FFA and PPA to study 
the neural correlates of visual awareness, as we will 
see later. 

3.0 THEORIES OF VISUAL 
CONSCIOUSNESS: 

WHERE DOES IT HAPPEN? 

So is it possible to say where along this cascade of 
neural activity consciousness is really happening? Is 
it possible to point to a particular set of neurons or a 
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BOX 6.1 Humans perceiving humans: neural correlates of biological motion 


Have you ever spotted a friend in a crowd simply 
because you recognized something in the way he or she 
moved, walked, or gestured? Is there something special 
in the way human beings perceive motion that is specific 
to our species, as opposed to other motion - like a wave 
breaking at the shoreline, a flag waving on a flagpole, or 
an airplane moving through the sky? 

Vision researchers have long asked these and related 
questions about human visual perception. An early prob¬ 
lem to address was: How do you investigate visual per¬ 
ception that is specific to human biological motion and 
not related to any other aspect of visual perception of the 
human form? A Swedish scientist developed an innova¬ 
tive way to do just that: In 1973, Gunnar Johansson pub¬ 
lished an article entitled 'Visual perception of biological 
motion and a model for its analysis' that has sparked dec¬ 
ades of studies on the topic of biological motion percep¬ 
tion. Dr. Johansson developed a method to attach small 
points of light to strategic places on his research assist¬ 
ant - shoulders, elbows, hands, knees, and feet - who 
was wearing dark-colored clothing. He filmed his assis¬ 
tant while moving about the dimly lit laboratory, creating 
stimulus sets that contained only the moving points of 
light (see Figure 6.15). 

Results of these early point-light motion studies and 
many more since have shown that, yes, we humans are 
quite skilled at perceiving each others' motions. The 
intriguing aspect of these studies is that no single light 
can give information about how the person is moving: it 
requires all the lights (or many of them at least). Follow¬ 
up studies have provided evidence that not just biologi¬ 
cal motion is perceived using point-light stimulus sets: 
observers can report the gender (Kozlowski & Cutting, 
1977; Mather & Murdock, 1994; Troje, 2002), emotional 
state (Dittrich, Troscianko, Lea, & Morgan, 1996), and even 
the size (Jokisch & Troje, 2003) of the "point-light person." 

Is biological motion perceived in the same brain areas 
where other types of motion are processed? Although 
there are many brain areas that are active for visual 
motion, evidence has emerged from several laborato¬ 
ries that biological motion activates a region in the pos¬ 
terior superior temporal sulcus (STSp) as one area that 
seems specifically sensitive to biological motion (Bonda, 
Petrides, Ostry, & Evans, 1996; Howard, Brammer, Wright 
et ah, 1996; Vaina et ah, 2001). 

In particular, the research of Professor Emily Grossman 
at the University of California, Irvine, has shed light on 
the brain areas and neural networks that underlie bio¬ 
logical motion (Grossman, Battelli, & Pascual-Leone, 
2005; Grossman & Blake, 2001, 2002a, 2002b; Grossman, 
Blake, & Kim, 2004; Grossman, Donnelly, Price et ah, 2000). 

But, is STSp specialized for processing human biologi¬ 
cal motion? Grossman and her colleagues are looking 
into this matter more closely, and recently reported that 
STSp was more responsive to point-light human motion 
as compared to point-light motion produced by mul- 
tijointed but nonhuman "creatures" (Figure 6.16) (Pyles, 
Garcia, Hoffman, & Grossman, 2007). Thus, they con¬ 
cluded that the STSp is indeed a region specialized for 
encoding human biological motion and not just complex 
motion or animation. 



FIGURE 6.15 The original figure from Johannson, (1973), 
showing the location of the points of light on a human (a) and 
how they looked when the human form was removed (b). 
Source : Johannson, 1973. 
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BOX 6.1 (Continued) 



FIGURE 6.16 Left panel: The stimuli used in Pyles et al. (2007) showing three fully illuminated "creatures" and their 
point-light animation, and the human and corresponding point-light animation. Right panel: Brain results for the "creatures" 
and human biological motion study. Note on the top panel that area STSp is more active for the human biological motion than 
the scrambled motion. Source : Pyles et al, 2007. 
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particular brain area and say. There it is ..., there is the 
place in my brain where I am experiencing the visual world 
in my mind ? 

It turns out that the answer is not so simple, but 
scientists are gathering important clues. Even if a per¬ 
son's eyes are closed or a person can no longer see 
because of damage to the eyes or to the LGN, it is still 
possible for a person to 'experience seeing' if electrical 
stimulation is applied to their primary visual cortex. 
In other words, it is possible to bypass stimulation of 


the retina and the LGN and induce visual experiences 
by directly stimulating area VI. 

Is it possible to bypass the primary visual cortex 
and induce a clear visual experience? We don't know 
the definite answer yet but, so far, the answer seems 
to be 'no'. Primary visual cortex seems to be impor¬ 
tant for our ability to consciously perceive any visual 
feature, while higher visual areas may have a more 
specialized role in perceiving certain visual features or 
objects (Tong, 2003). As we will see in the remainder 
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More aware 


Less aware 


More aware 


(a) Hierarchical model 



Complete awareness 
"'X requires feedback signals 


(b) Interactive model 


FIGURE 6.17 Hierarchical 
and interactive models of visual 
awareness, (a) In the hierarchical 
model, with each step further in 
visual processing, awareness is 
more likely to result from that 
processing, (b) In the interactive 
model, feedback signals from 
later processing areas to earlier 
processing areas are needed to 
attain awareness. At present, it is 
not clear which of the two mod¬ 
els best describes the way brain 
activity results in visual aware¬ 
ness. Source : Frank Tong, with 
permission. 


of this chapter, different cortical visual areas seem to 
play different roles in our conscious visual experience. 
An emerging view is that many of the same brain areas 
and neurons involved in processing specific kinds of 
visual stimuli, such as orientation, motion, faces, or 
objects, are also involved in representing these types 
of stimuli in consciousness. Many neurons are more 
active when a person is conscious of seeing a stimulus 
than when the stimulus is shown but fails to reach con¬ 
sciousness. So far, there doesn't seem to be any single 
area in the brain that is solely responsible for conscious¬ 
ness. Instead, many brain areas seem to work together 
to achieve this remarkable feat. Brain areas involved 
in attentional processing are also important for the 
ability to perceive and respond to visual and other sen¬ 
sory inputs, a topic that will be covered in Chapter 8. 

3.1 Hierarchical and interactive theories 
of vision 

According to one theory of visual consciousness, called 
the hierarchical theory (Crick and Koch, 1995; Rees 
et ah, 2002), consciousness is organized in a hierarchi¬ 
cal fashion with increasingly higher visual areas being 
more closely related to our internal conscious experi¬ 
ence (Figure 6.17a). This theory is consistent with the 
notion that higher visual areas respond to more com¬ 
plex stimuli, such as entire objects, and can integrate 
information about many visual features, which are 
processed in early visual areas. However, if this is the 
case, how is it that we can be aware of specific visual 
features or very fine spatial details, information that is 
best represented in early visual areas such as VI? 


The interactive theory of visual consciousness empha¬ 
sizes a different idea. It turns out that the signals entering 
the brain do not simply travel up the visual hierarchy: 
higher visual areas send feedback signals back down to 
early visual areas, especially to area VI (Bullier, 2001). 
There are as many feedback projections in the visual system, 
neurons projecting from higher visual areas to lower vis¬ 
ual areas, as there ar e feedforward projections. According to 
the interactive theory, once a stimulus is presented, feed¬ 
forward signals travel up the visual hierarchy, activating 
many neurons in its path, but this feedforward activity 
is not enough for consciousness. Instead, high-level areas 
must send feedback signals back to lower-level areas 
where the feedforward signals came from, so that neural 
activity returns full circle, forming a neural circuit (Figure 
6.17b) (Pollen, 1999; Lamme and Roelfsema, 2000). Why 
might this combination of feedforward-feedback sig¬ 
nals be important for awareness? This may be because 
higher areas need to check the signals in early areas 
and confirm if they are getting the right message, or 
perhaps to link neural representations of an object to 
the specific features that make up the object. 

So far, it is not clear which theory will prove true 
and odds are that both theories capture some parts of 
the bigger picture. An emerging view is that our con¬ 
scious experience may reflect the distributed pattern 
of brain activity involving many visual areas, a kind of 
dialogue between neurons in early visual areas, includ¬ 
ing VI, and high-level areas such as those in the ven¬ 
tral temporal cortex and the parietal lobe. That said, 
the contribution of each brain area may be somewhat 
different or in some cases, unique. The next parts of 
this chapter will describe the highlights of this story. 
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FRONTIERS OF COGNITIVE NEUROSCIENCE 


Eye movements and visual fading 



FIGURE 6.18 (a) Susana Martinez-Conde, Laboratory of Visual Neuroscience, Barrow Neurological Institute, Phoenix, AZ, USA. 

(b) Stephen L. Macknik, Laboratory of Behavioral Neurophysiology, Barrow Neurological Institute, Phoenix, AZ, USA. 


Visual fading is a phenomenon in which an unchanging 
visual stimulus disappears from perception. The neu¬ 
ral substrate of visual fading is neural adaptation, by 
which visual neurons decrease or cease their responses 
to unchanging visual stimuli. 

Visual fading and neural adaptation 

Nervous systems have evolved to detect changes in 
the environment. We can detect stationary objects only 
because the images projected onto our retinas are never 
stationary for long: even during visual fixation, our eyes 
are never still. 

To experience visual fading, we must remove the 
effects of eye movements, so that the visual images of 
unchanging objects remain stable on the retina, leading 
to the adaptation of neuronal responses. 

Visual fading under retinal stabilization conditions 

To achieve perfect retinal stabilization, we must eliminate 
all eye movements. This is complicated because our eyes 
are in constant motion (Figure 6.19(a)). Even when we 
fixate our gaze precisely on an object of interest, we nev¬ 
ertheless constantly produce small involuntary eye move¬ 
ments, called fixational eye movements (Figure 6.19(b)). 
These tiny eye movements are usually sufficient to pre¬ 
vent neural adaptation and visual fading. 

Holding a subject's eyes physically stationary (with 
paralyzing drugs, for instance) is technically very 
difficult, and potentially hazardous to the subject's 
health. Therefore, retinal stabilization studies generally 
have used an alternative approach: to shift the visual 


stimulus in such a way that all eye movements are 
effectively nulled. That is, the visual stimulus must 
move in the same direction, speed, and amplitude as 
the eye, so that the retinal image remains stable despite 
eye movements. 

Early stabilization experiments reported that images 
faded only after several seconds of stabilization. However, 
Coppola and Purves (1996) more recently showed that the 
images of retinal vascular shadows (which are extremely 
stable with respect to the eye) disappear in less than 
80 msec, suggesting that normal visual processing 
entails a very rapid mechanism for image creation and 
erasure. 

Visual fading without retinal stabilization 

Although perfect retinal stabilization is most easily 
achieved under laboratory conditions, fading of stationary 
objects, especially in the visual periphery, occurs quite 
often in everyday vision. In 1804, Swiss philosopher Ignaz 
Paul Vital Troxler noted that deliberately focusing the gaze 
on something causes stationary images in the surrounding 
region gradually to fade away (Figure 6.19(c)). 

Martinez-Conde et al. (2006) showed that Troxler fading 
occurs due to the suppression of microsaccades (the larg¬ 
est and fastest type of fixational eye movement) during 
precise and sustained fixation. Microsaccade production 
dropped just before a visual target vanished, indicating 
that a lack of microsaccades leads to neural adaptation 
and fading. Conversely, microsaccade production peaked 
right before the target reappeared. These results dem¬ 
onstrated for the first time that microsaccades have a 










FIGURE 6.19 Eye movements and visual fading, (a) An observer views a picture (left) while eye positions are monitored (right). 
The eyes jump, seem to fixate or rest momentarily, producing a small dot on the trace, then jump to a new region of interest. The 
large jumps in eye position illustrated here are called saccades. However, even during fixation, or "rest" times, eyes are never still, but 
continuously produce fixational eye movements: drifts, tremor, and microsaccades. Source: Yarbus, 1967. (b) Cartoon representation of 
fixational eye movements in humans and primates. Microsaccades (straight and fast movements), drifts (curvy slow movements), and 
tremor (oscillations superimposed on drifts) transport the visual image across the retinal photoreceptor mosaic. Source: Martinez-Conde 
& Macknik, 2007. (c) Troxler fading. In 1804, Swiss philosopher Ignaz Paul Vital Troxler discovered that deliberately fixating on some¬ 
thing causes surrounding stationary images to fade away. To elicit this experience, stare at the central dot while paying attention to the 
surrounding pale ring. The ring soon vanishes, and the central dot appears set against a white background. Move your eyes, and it pops 
back into view. Source: Modified from Martinez-Conde, Macknik, & Hubei, 2004. (d) Microsaccades bring about visibility when people 
are fixing their gaze. Subjects stared at a small spot in the center of a computer screen, causing a static peripheral target to vanish from 
their view and then reappear. Just before the target vanished, the viewers' microsaccades became sparser, and right before it reappeared, 
these eye movements became more frequent. Source: Martinez-Conde and Macknik, 2006. 


critical role in driving visibility during visual fixation 
(Figure 6.19(d)). 

Implications for visual consciousness 

Fixational eye movements counteract the visual fad¬ 
ing of stationary objects and drive their visibility dur¬ 
ing fixation. Thus, fixational eye movements can help 
constrain the spatiotemporal characteristics of stim¬ 
uli that are consciously visible. Moreover, the neural 
responses triggered by fixational eye movements along 
the visual pathway must encompass the neural code for 
visibility. 
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4*0 BRAIN AREAS NECESSARY 
FOR VISUAL AWARENESS: 
LESION STUDIES 

In the previous section, we learned about how the 
neurons in different visual areas respond best to 
certain visual features, objects, or spatial informa¬ 
tion. Such findings from single-neuron recordings or 
human neuroimaging may indicate that a particular 
brain area has a role in processing a certain type of 
stimulus, but cannot reveal whether that brain area is 
actually necessary for perceiving that stimulus. Perhaps 
that brain area could be removed without causing 
any troubles with perception. For example, if multi¬ 
ple brain areas are involved in processing a particular 
visual feature (e.g. motion), then damage to any single 
brain area might not impair the ability to perceive that 
feature because other intact brain areas would be able 
to compensate for the loss. 

Brain lesion studies are important for under¬ 
standing what brain areas may be necessary for cer¬ 
tain kinds of visual awareness - awareness of color, 
motion, faces, objects, or the capacity to be aware of 
seeing anything at all! Brain lesions may be performed 
experimentally, in animal studies, or may be investi¬ 
gated in humans who have suffered from unfortunate 
injury to certain parts of the brain, which may result 
from strokes, tumors, trauma, or neurodegenerative 
diseases. Visual deficits resulting from damage to 
certain parts of the visual system can be very debili¬ 
tating. However, by studying these patients, it may 
be possible to understand the neural causes of their 
impairment, which may inform scientists about brain 
function and eventually lead to new ways to help treat 
such impairments. 


4*1 Consequences of damage to early 
visual areas 

Different visual deficits can result from neural dam¬ 
age at different levels of the visual processing hierar¬ 
chy. Damage to the retina or optic nerve of one eye can 
result in monocular blindness - the loss of sight from 
one eye. Damage to the LGN, the optic radiations that 
travel to VI, or VI itself, can lead to loss of vision in the 
contralateral visual field (see Figure 6.7). Damage to a 
small part of VI can lead to a clearly defined scotoma, 
a region of the visual field where perception is lost. 
The first retinotopic maps of VI were actually con¬ 
structed by mapping the trajectory of bullet wounds of 


soldiers injured in the Russo-Japanese war and World 
War I; there was a clear relationship between the loca¬ 
tion of each case of scotoma and the part of VI that was 
injured (Inouye, 1909; Holmes, 1918). 


4. 1.1 VI and blindsight 

Do lesions to VI lead to a complete loss of visual func¬ 
tion? In 1965, researchers investigated this question in 
a rhesus monkey named Helen, after the majority of 
visual cortex was removed. For the initial 19 months, 
Helen displayed behavior that suggested she was 
completely blind. Gradually however, it seemed that 
her vision was returning. In time, she could navigate 
among objects in a crowded room and even reach out 
to catch a passing fly. Initially, she would only look 
at and reach for objects when they moved. Then, as 
time passed, she responded to flashing lights, then a 
stationary light source, and, finally, a stationary dark 
object against a light background (Humphrey and 
Weiskrantz, 1967; Humphrey, 1974). This is an excellent 
example of how recovery of function can occur after 
brain injury. Helen was able to locate salient objects, 
but seemed unable to recognize them. 

Was Helen aware of what she saw, or was she able 
to perform visually guided actions despite a lack of 
visual awareness? It is difficult to ask an animal if it 
is conscious of something or not, but a recent study of 
monkeys with unilateral VI lesions suggests that they 
might not be aware of what they see. In this study, 
monkeys were able to report the locations of objects in 
their 'blind' hemifield accurately if they were forced to 
make a choice between two options. However, if they 
were given the choice of reporting whether an object 
was presented or not and an object was sometimes 
presented in the good hemifield, in the blind hemi¬ 
field, or not at all, they would fail to report objects 
presented in the blind hemifield (Cowey and Stoerig, 
1995; Stoerig et al., 2002). It is as if these objects were 
not 'seen'. 

Interestingly, humans with VI lesions may show 
similar above-chance performance, even though they 
report a lack of visual experience in their blind hemi¬ 
field. Patient DB suffered extreme migraines because 
of a venous tumor lodged in his right calcarine cortex 
(VI). Surgical removal of the tumor led to the loss of 
his right primary visual cortex. As you might expect, 
this procedure left him practically blind in his left vis¬ 
ual field. Interestingly, however, when he was system¬ 
atically tested in his blind hemifield, his performance 
suggested otherwise. 
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Weiskrantz et al. (1974) reported that when DB was 
asked to point to a target in his blind visual field, DB 
claimed he could see nothing at all, yet he could point 
quite accurately to the location of the light source. He 
could also accurately report whether a stick was angled 
vertically or horizontally. This ability to perform vis¬ 
ual tasks at above-chance levels, despite the patient's 
reports of lacking any visual impressions, is known 
as blindsight. In different experiments, his task was to 
discriminate between Xs and Os. His performance was 
quite accurate and improved as a function of the size 
and duration of the stimuli. During all these experi¬ 
ments DB insisted that he saw nothing. Interestingly, 
when pressed, he reported sometimes having the feel¬ 
ing of 'smoothness' or 'jaggedness' in discriminat¬ 
ing the Xs and Os, but denied that these feelings were 
associated with any visual experience. So how is this 
possible? How can DB point to or discriminate visual 
stimuli without being able to see them? 

These findings suggest that there can be disso¬ 
ciations between visual processing in the brain and a 
person's subjective awareness - sufficient information 
is reaching DB's visual system to allow him to make 
forced-choice discriminations, but this information 
is not sufficient to support awareness (Weiskrantz, 
1986). However, one concern is whether visual aware¬ 
ness is completely absent in blindsight. Patients might 
be reluctant to report weak visual impressions that are 
nonetheless sufficient for making forced-choice dis¬ 
criminations. Similar effects can occur in normal sub¬ 
jects under near-threshold conditions. Some blindsight 
patients report residual impressions of salient mov¬ 
ing stimuli, which they describe as 'black moving on 
black', but normally they report seeing nothing when 
shown static stimuli. Vision in blindsight is severely 
degraded, but not always completely absent. 

Intact extrastriate cortex may be crucial for blind- 
sight, as patients who have had an entire hemisphere 
removed show little evidence of residual visual abili¬ 
ties (Faubert et al., 1999). Although the pathway from 
retina to LGN to VI provides the vast majority of visual 
input to cortex, several alternative subcortical pathways 
project to extrastriate areas, bypassing VI. Single-unit 
recordings in monkeys indicate that visual informa¬ 
tion can still reach extrastriate areas after VI has been 
lesioned. Although firing rates are reduced, a substan¬ 
tial proportion of neurons in motion area MT and V3 
remain selectively responsive to visual stimuli (Rodman 
et al., 1989; Girard et al., 1991). Recent neuroimaging 
studies show that unperceived stimuli presented to 
the blind hemifield still evoke robust fMRI responses 
from motion-sensitive area MT, color-sensitive area V4 


and regions involved in object perception (Goebel et al., 
2001). Thus, considerable stimulus selectivity is main¬ 
tained in extrastriate cortex, yet this activity appears 
insufficient to support awareness in the absence of VI. 
This is consistent with the predictions of interactive 
models. However, it remains possible that extrastriate 
signals are too weak or degraded to support conscious 
perception but sufficient to support forced-choice 
discrimination. 


4.2 Extrastriate lesions - damage outside 
area V1 

Lesions to VI can eliminate almost all visual awareness 
in the corresponding areas of visual space, even though 
sufficient visual information seems to be reaching 
extrastriate areas to support blindsight behavior. This 
in itself informs us as to the role of VI in the mediation 
of visual awareness. So what happens when extrastri¬ 
ate areas are damaged? See Figure 6.20 for a schematic 
map of visual areas and corresponding deficits. 

4.2.1 Motion blindness 

Perhaps because we are so sensitive to seeing motion, 
it is very rare for brain damage to lead to a complete 
loss of motion perception. However, there is a strik¬ 
ing example of one patient who can no longer perceive 
motion after suffering strokes leading to large bilateral 
lesions that encompassed area MT and extensive sur¬ 
rounding areas. For this patient, the world appeared 
to be a series of still snapshots, like living in a strobe- 
lit world. Simple tasks like crossing the street became 
dangerous, because she could not tell how fast the cars 
were approaching (Figure 6.21(a)). Even pouring a cup 
of coffee became a challenge, since she couldn't tell how 
fast the liquid was rising, so the cup would overflow. 

Other studies have found that smaller lesions just 
to area MT usually lead to more moderate deficits in 
the ability to perceive motion and both patients and 
animals may also show considerable recovery over 
time (Plant et al., 1993; Pasternak and Merigan, 1994). 
So it seems that area MT is very important for motion 
perception, but other visual areas can contribute to 
motion perception even when MT is damaged. 

4.2.2 Cortical color blindness 

Damage to ventral area V4 can lead to cortical color 
blindness or what is sometimes called achromatopsia 
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(Meadows, 1974a; Bouvier and Engel, 2006). Patients 
report that the world appears to be drained of color, 
almost like shades of gray, perhaps like the illustration 
in Figure 6.21(b). These patients can still perceive the 
boundaries between colors, but have difficulty with 
identifying the colors themselves. Achromatopsia is 
typically associated with lesions that include area V4 
and possibly regions just anterior to area V4. Damage 




FIGURE 6.20 Visual deficits and brain areas. The areas of the 
brain in which damage can result in the associated visual deficit. Here, 
the areas are only shown on one hemisphere, although for some defi¬ 
cits like motion blindness, damage to both hemispheres is required. 
Source: Frank Tong, with permission. 


to one hemisphere can even lead to selective loss of 
color perception in the contralateral visual field. 

4.3 Damage to ventral object areas 

4.3.1 Visual agnosia 

Patients with visual agnosia have difficulties with recog¬ 
nizing objects because of impairments in basic percep¬ 
tual processing or higher-level recognition processes. 
Such patients can still recognize objects by using other 
senses such as touch, hearing, or smell, so the loss of 
function is strictly visual. The word agnosia can be trans¬ 
lated from Greek as meaning To lack knowledge of', so 
visual agnosia implies a loss of visual knowledge. 

Here, we will discuss three types of visual agnosia: 
apperceptive agnosia, associative agnosia, and proso¬ 
pagnosia. Patients with apperceptive agnosia can still 
detect the appearance of visually presented items, but 
they have difficulty perceiving their shape and cannot 
recognize or name them. These patients usually fail at 
shape-copying tests and may have difficulty copying 
very simple shapes, such as a circle, square, or perhaps 
even a single tilted line. Carbon monoxide poisoning 
is a frequent cause of apperceptive agnosia, as this can 
lead to profuse damage throughout the occipital lobe. 

Remarkably, some apperceptive agnosics show evi¬ 
dence of unconscious visual processing of visual fea¬ 
tures they cannot consciously perceive. Goodale et al. 
(1991) tested an apperceptive agnosic patient, DF, who 
had difficulty reporting the orientation of simple lines 
or real objects. When asked to report the orientation 
of a narrow slot cut into the face of a drum, she was 
unable to report the angle of the slot and made many 
errors (Figure 6.22, top panel; the drum was rotated 



(a) (b) 


FIGURE 6.21 Color and motion blindness, (a) Damage to motion area MT in both hemispheres can lead to a 
loss of motion perception: akinotopsia. Patients describe seeing multiple still frames instead of smooth motion. This 
can make simple tasks like crossing the road challenging and dangerous, (b) Damage to color areas in only one 
hemisphere of the cortex can result in a loss of color perception to one side of visual space. Cortical color blindness 
is called achromatopsia. Source: Frank Tong, with permission. 
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FIGURE 6.22 She can do it, but can't report it. Results from 
subject DF. Each line represents one of DF's attempts at either 
matching the orientation of a mail slot or actually posting a letter 
into it. The top panel shows DF's attempts at matching the orienta¬ 
tion of the slot to the card (no execution of action). Here, the lines 
are distributed around the circle suggesting that DF had trouble in 
perceiving the orientations. However, the lower panel shows DF's 
accuracy in posting the card into the oriented slot, adjusted so that 
each correct orientation is vertical. Because the lines are all grouped 
around vertical, this suggests that DF is accurate when executing 
the action of posting. Patient DF can post the letters with no prob¬ 
lem; her hand knows what to do. But when asked to match the ori¬ 
entation of the slot, DF performs very badly. Source : Frank Tong, 
with permission. 

to a new orientation on each trial). However, when 
asked to post a card through the slot, she could do so 
with remarkable accuracy (Figure 6.22, lower panel) 
and was even surprised by her own ability to do so. 
Surprisingly, when she was asked just to hold the let¬ 
ter by her side and rotate it to match the angle of the 
slot, her performance was poor once again and she 
reported that the slot seemed dess clear' than when 
she was allowed to post the card. What can account 
for this behavioral dissociation between DF's ability to 
report the angle of the slot and to perform a visually 
guided action? 

Patient DF provides strong evidence to suggest that 
there are separate pathways for processing 'what' an 
object is and 'where' it is with respect to performing 
a visually guided action. According to Goodale and 
Milner, patient DF has damage to the ventral visual 
pathway but intact processing in the dorsal pathway, 
a claim that has recently been supported by brain 


imaging studies (James et al., 2003). They propose that 
the dorsal system is not only responsible for process¬ 
ing 'where' objects are, but also 'how' actions can be 
performed toward a particular object, such as point¬ 
ing or reaching for that object. Apparently, visual 
processing in the dorsal system is not accessible to 
consciousness - the patient can't report the orientation 
of the slot - yet the dorsal system can guide the right 
action. Complementary studies of patients with optic 
ataxia have revealed the opposite pattern of visual 
deficits, indicating a double dissociation between con¬ 
scious visual perception and visually guided action. 
Optic ataxia typically results from damage to the pari¬ 
etal lobe, which is part of the dorsal pathway. These 
patients can perceive visual orientations and recog¬ 
nize objects well, but have great difficulty performing 
visually guided actions. 

Associative agnosia refers to the inability to recog¬ 
nize objects, despite apparently intact perception of 
the object. For example, when asked to copy a simple 
picture, patients with associative agnosia manage to 
do a reasonable job, especially if given enough time. 
In comparison, this task would almost be impossible 
for an apperceptive agnosic. 

Dr. Oliver Sacks described a patient who, as a 
result of brain injury, 'mistook his wife for a hat'. 
The patient had great difficulty identifying objects, 
even though his vision was otherwise normal and he 
could describe the features of what he saw. When pre¬ 
sented with a rose from Dr. Sacks' lapel, the patient 
described it as 'a convoluted red form with a linear 
green attachment', but only upon smelling it did he 
realize that it was a rose. When his wife came to meet 
him at the doctor's office, he accidentally reached for 
her head when he wanted to retrieve his hat from the 
coat rack (Sacks, 1985). 

Associative agnosia usually results from damage 
to the ventral temporal cortex. Typically, associative 
agnosics will have difficulty recognizing a variety of 
objects, especially objects that are all from a single cat¬ 
egory, such as faces. However, as we will see below, 
this is often but not always the case. 

Although most patients with visual agnosia will 
have difficulty with recognizing both faces and objects, 
there are some remarkable exceptions that have been 
reported. Patients with prosopagnosia are still able to 
recognize objects well, but have great difficulty recog¬ 
nizing or telling apart faces (Bodamer, 1947; Meadows, 
1974b). Deficits can be severe; some prosopagnosic 
patients can no longer recognize close family members 
or friends and, instead, must rely on other cues such as 
the person's voice or clothing to recognize that person. 
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Some patients can no longer recognize their own face 
in photos or even in the mirror. 

Is prosopagnosia really due to a specific impair¬ 
ment in face recognition? Maybe face recognition is 
just more difficult than other forms of object recogni¬ 
tion. After all, faces are very visually similar to one 
another - every face has the same basic shape and con¬ 
figuration of features, whereas objects are much more 
distinct from one another (see Box 6.2 for discussion). 

Studies have revealed a few patients who can dis¬ 
criminate between subtle differences in objects but can 
no longer distinguish between faces. For example, a 
prosopagnosic farmer became very poor at recognizing 
human faces but could still recognize the different sheep 
in his flock (McNeil and Warrington, 1993). Another 
patient could no longer recognize upright faces accu¬ 
rately and was actually better at recognizing upside- 
down faces, which is just the opposite of normal subject 
performance (Farah et al., 1995). What might be the rea¬ 
son for this? According to one theory, the brain may 
have specialized mechanisms for processing upright 
faces, whereas upside-down faces are processed by a 
general object recognition system (see the face inver¬ 
sion demonstration in Figure 6.23). So, if the face recog¬ 
nition system is damaged in a prosopagnosic patient, 
then this system might lead to automatic errors when 
the patient is presented with an upright face. However, 
upside-down faces would not be automatically 


processed by this system, thereby allowing the object 
recognition system to take over the task. 

Perhaps the strongest evidence for some separation 
between face and object recognition systems comes 
from a study of a very unusual object agnosic patient, 
CK, who was very impaired at recognizing objects but 
could recognize upright faces just as well as normal 
subjects (Moscovitch et al., 1997). Remarkably, if the 
faces were turned upside-down, CK became severely 
impaired, performing six times worse than normal 
participants under these conditions. Apparently, CK 
has an intact system for processing upright faces, but 
the system responsible for processing objects and 
upside-down faces is badly damaged. Taken together, 
evidence from patient CK and prosopagnosic patients 
provides evidence of a double dissociation between 
the visual processing of upright faces as compared to 
objects and upside-down faces. 

What type of brain damage leads to prosopagno¬ 
sia? Prosopagnosia can result from bilateral dam¬ 
age around the regions of the lateral occipital cortex, 
inferior temporal cortex, and the fusiform gyrus 
(Meadows, 1974b; Bouvier and Engel, 2006). In some 
cases, unilateral damage to the right hemisphere may 
lead to this impairment. Because lesions are usually 
quite large and might damage fiber tracts leading to a 
critical brain region, it is difficult to identify a precise 
site. Nonetheless, the brain lesion sites associated with 


BOX 6.2 How are faces and objects represented in the brain? 


There are two primary theories about how the visual sys¬ 
tem manages to process and recognize objects and faces. 
The brain could either do it in a modular fashion, with 
distinct modules for processing faces, or the processing 
could be done in a distributed way across multiple areas 
of the ventral temporal cortex. According to the modular 
view, object perception is broken down into neural mod¬ 
ules, specific areas of the brain that specialize in process¬ 
ing a particular object category. Research suggests that 
the fusiform face area (FFA) might be a specialized mod¬ 
ule for the processing and recognition of upright faces 
(Kanwisher et al., 1997; Tsao et al., 2006). In addition, an 
area that responds to the presentation of places (e.g. a 
house) seems also to be a specialized module (Epstein 
and Kanwisher, 1998). This area has become known as the 
parahippocampal place area (PPA). This trend for modu¬ 
lar representation of objects does not span every object; 
in fact, it is primarily observed only in the two above- 
mentioned cases. For example, there is not a banana or 
shoe area in the human brain. 

An interesting twist on the modular hypothesis is the 
expertise hypothesis, which proposes that the so-called 
fusiform face area is actually specialized for expert object 


recognition (Gauthier et al., 2000). We probably spend 
more time looking at faces than at any other object (espe¬ 
cially if you include watching faces on TV), just stop a 
moment and think about how much information you can 
get from all the subtle changes in someone's face when 
you're talking to him or her and you soon realize you are 
indeed a face expert. It has been proposed that the FFA 
is responsible for the recognition process of any object 
we are 'experts' at. Research shows that, while looking 
at pictures of birds, bird experts show somewhat stronger 
activity in the FFA than do non-bird experts (Gauthier et al, 
2000). Whether the FFA is specific for face recognition or any 
object of expertise, both cases involved a specialized struc¬ 
ture that is distinct from regions of the ventral temporal 
cortex involved in object processing. Look at the two faces 
in Figure 6.23 - do you notice anything strange? Now turn 
the book upside down and look again - now do you see it? 
This is an example of the face inversion effect. When faces 
are upside down we are really bad at identifying them. This 
is an example of how specialized we are at face perception. 

The other hypothesis is that the brain processes faces 
and objects in a distributed way across multiple areas in 
the ventral pathway. A study by Haxby and colleagues 
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BOX 1.1 (Continued) 


FIGURE 6.23 The face inversion effect. Demonstration of how bad we are at recognizing upside down faces. Look at the two 
pictures of Barack Obama. Do you notice anything strange? Now turn the page upside down and look again. Now you should see that 
one of the pictures has been severely distorted. This effect, called the face inversion effect, demonstrates just how specialized our visual 
system is for processing upright faces. 


demonstrated that regions outside of the FFA still show 
differential responses to faces, as compared to other types 
of stimuli (Haxby et ah, 2001). Even if the response differ¬ 
ence between faces and different objects is quite small in 
many of these areas, there is enough information across 
all of these regions outside of the FFA to tell the differ¬ 
ence between faces and objects. However, neuropsy¬ 
chological evidence of double dissociations between 
face recognition and object recognition are difficult to 


explain according to this account. One possible resolu¬ 
tion is that the activity of highly face-selective neurons 
in regions such as the FFA may be important for telling 
apart subtle differences between individual faces, but 
that more distributed activity patterns outside of these 
highly face-selective regions are enough for telling apart 
basic differences between faces and objects and perhaps 
more obvious differences between faces (e.g. male versus 
female, young versus old). 


prosopagnosia appear to encompass the fusiform face 
area and extend much more posteriorly. 


4.4 Damage to dorsal parietal areas 

Damage to the posterior parietal lobe (or superior tem¬ 
poral gyrus) can lead to a striking global modulation 
of visual awareness called neglect, in which a patient 
completely ignores or does not respond to objects in 


the contralateral hemifield (Driver and Mattingley, 
1998). Patients with right parietal damage may ignore 
the left half of the visual field, eat just half of the food 
on their plate, or apply make-up to just half of their 
face. They may also ignore sounds or touches coming 
from their left. 

This syndrome can resemble a disorder of visual 
perception. However, neglect happens in the absence 
of damage to the visual system and can involve mul¬ 
timodal deficits, including motor and tactile deficits. 
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Moreover, when patients are instructed to attend to 
the neglected field they can sometimes respond to 
these stimuli (Posner, 1980). So this syndrome is more 
a deficit of attention than an inability to perceive stim¬ 
uli. Without specific cuing, patients find it difficult to 
perceive objects in their neglected field. We will not 
spend much time delving into the many interesting 
facets of visual attention as an entire chapter is dedi¬ 
cated to it (see Chapter 8). 

Bilateral lesions to parietal areas can lead to a much 
more profound deficit called Balint's syndrome, which 
is primarily a disruption of spatial attention. It can be 
characterized by three main deficits: (1) optic ataxia, 
the inability to point to a target; (2) ocular apraxia, the 
inability voluntarily to shift gaze; and (3) simultanag- 
nosia, the inability to perceive more than one object 
in the visual field simultaneously, even when the 
objects occupy a common region of space. People with 
Balint's syndrome can often appear blind because, in 
fact, they can only focus on one region of visual space 
or one object at a time and find it hard voluntarily to 
shift their attention to different locations. These results 
again highlight the importance of attention-related 
brain areas in supporting visual awareness. 

5*0 LINKING BRAIN ACTIVITY AND 
VISUAL EXPERIENCE 

At any given moment, millions of neurons are firing 
in the visual part of your brain. The activity of some of 
these neurons is probably closely linked to what you are 
consciously perceiving here and now, while the activity 
of other neurons may be far removed from immediate 
experience. If scientists could somehow isolate the neu¬ 
rons that closely reflect a person's conscious experience, 
would this reveal which neurons or brain areas are cen¬ 
tral to consciousness and which are not? This search is 
for the neurons that correlate with changes in perceptual 
awareness or consciousness; hence it has been called 
the search for the neural correlates of consciousness or the 
NCC for short (Crick and Koch, 1995). 

The first challenge involves telling apart stimulus- 
driven activity from activity linked to awareness. Not 
all visual responses are conscious, as can be told from 
the fact that many visual areas respond well to stimuli 
even when an animal is anesthetized. 

How then is it possible to measure awareness-related 
activity and separate this from any possible unconscious 
activity that is driven by the visual stimulus itself? There 
are many tools scientists can use for this purpose. The 


next section will discuss some of the most popular and 
useful methods for isolating neurons whose activity cor¬ 
relates with changes in visual awareness. 

5T Multistable perception 

Have you ever been in the dark, perhaps lying in bed, 
staring at some strange shape across the room? At one 
moment it might look like a person, then like some 
strange static animal, then like a rock statue. After rack¬ 
ing your brain to identify this mysterious object, you 
finally hit the light switch and lo and behold it's only 
the dim light catching your jacket on the chair. In this 
example, when vision was difficult and ambiguous, per¬ 
ception did something uncommon - it faltered or alter¬ 
nated between different things. This is an example of 
multistable perception : the jacket on the chair (the physi¬ 
cal stimulus) did not change while your perception 
of it did! This kind of situation is a valuable tool, as it 
enables scientists to study changes in visual awareness 
independent of any changes in the visual stimulus. 

There are many examples of multistable patterns or 
ambiguous figures that scientists can use to investigate 
these neural correlates of consciousness. Patterns that 
primarily have only two primary interpretations are 
called bistable patterns (Figure 6.24). Try to see if you 
can perceive both interpretations of each ambiguous 
figure, the face and the vase and the two views of the 
Necker cube. Bistable perception can still occur if you 
keep your eyes fixed on a single point in the middle of 
the stimulus. In this case, the image on your retinas - 
and hence the stimulus-driven activity - is pretty 
much constant over time, while your perception fluc¬ 
tuates. By looking for brain areas that show activity 
changes correlated with perceptual changes, scientists 
can identify the neural correlates of consciousness. 

5.2 Binocular rivalry: what you see is what 
you get activated 

One of the most powerful and best-studied examples 
of bistable perception is a phenomenon called binocular 
rivalry. When two very different patterns are shown, one 
to each eye, because they are so different the brain can¬ 
not fuse them together like it would normally do. What 
then happens is quite striking: awareness of one pattern 
lasts for a few seconds, then the other pattern seems 
magically to appear and wipe away the previously vis¬ 
ible pattern. It is like the two patterns are fighting it out 
in the brain for your perceptual awareness! If you can 
get your hands on a pair of red-green filter glasses then 
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FIGURE 6.24 Bistable figures, (a) After 
looking at this figure for a while you will notice 
that there are two interpretations. One is a cen¬ 
tral vase; the second one silhouettes of two faces 
looking in at each other. This image is bistable: 
while you look at it your perception will alter¬ 
nate between the vase and the faces, (b) This 
wireframe cube, typically known as the Necker 
cube, has two equally likely spatial interpreta¬ 
tions. Perception tends to alternate between 
the configuration of the closest side projecting 
upward and the closest side projecting down¬ 
ward. Like the vase and silhouettes above, this is 
bistable. This bistability allows a dissociation of 
low level stimulation and awareness. The physi¬ 
cal pattern does not change, but your awareness 
of it does! 



FIGURE 6.25 Binocular rivalry (a) If you have a pair of red-green filter glasses, this image should produce 
binocular rivalry Your awareness should alternate back and forth between vertical and horizontal striped patterns, 
(b) If you don't have a pair of red-green glasses, try cross-fusing these two patterns. To cross-fuse, you need to 
cross your eyes so the two patterns line up and appear to be on top of one another. The surrounding squares can 
help you do this. Line up the square outline and the bull's-eye dot in the middle, (c) Schematic of how Tong et al. 
(1998) used red-green glasses to attain binocular rivalry in the fMRI scanner and the FFA and PPA where they 
found activity that correlated with awareness. Source: Frank Tong, with permission. 


you can experience binocular rivalry by viewing Figure 
6.25(a). Otherwise try cross-fusing the two patterns in 
Figure 6.25(b). (To cross-fuse, try letting your eyes cross 
so the two patterns appear one on top of the other. The 
dots in the center and surrounding square should fuse. 


If these demos work for you, you should see one pat¬ 
tern, while the other is completely invisible, almost like 
closing one eye and then the other.) 

What happens in the brain during binocular rivalry? 
Tong et al. (1998) tackled this problem by focusing on 
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FIGURE 6.26 The stimuli and data from Tong et al. (1998). (a) Top panel shows the binocular rivalry condition. 
Subjects experienced first the face, then the house, then the face, etc. Lower panel shows a control condition with no bin¬ 
ocular rivalry; the images were switched on and off the screen, (b) Top panel shows the fluctuations in activity in both 
the FFA (blue) and PPA (red) during binocular rivalry. When each image became perceptually dominant, activity in the 
corresponding area increased. Lower panel shows the neural response in the same areas to the control condition with no 
binocular rivalry. The alternations in activity in both conditions were around the same size. Source : Tong et al., 1998. 


two category-selective areas in the ventral temporal 
lobes, the FFA and the PPA. They used red-green filter 
glasses to present a face to one eye and house to the 
other eye while measuring fMRI activity from these 
brain areas (Figure 6.25(c)). In this study, participants 
might first perceive the house, then flip to the face 
and then back again - as is typical of binocular rivalry 
(Figure 6.26(a)). Remarkably, the FFA was active only 
when subjects reported that they saw the face. Likewise, 
the PPA was active only when the participants reported 
that they saw the picture of the house (Figure 6.26(b)). 
Next, the researchers tested physical alternations 
between two the pictures, switching one picture on 
while switching the other off. The resulting stimulus- 
driven responses in the FFA and PPA were about the 
same strength as those measured during binocular 
rivalry and, surprisingly, no stronger. It seems that the 
activity in these brain areas closely mirrors what the 
observer perceives during rivalry and doesn't reflect 
the temporarily suppressed stimulus that is still activat¬ 
ing the retina. 

Other brain imaging studies have found that activ¬ 
ity at earlier stages of visual processing is closely 


linked to what the observer perceives during binocular 
rivalry. For example, strong awareness-related modu¬ 
lations have been found in human VI (Polonsky et al., 
2000; Tong and Engel, 2001), and even in the lateral 
geniculate nucleus (Haynes et al., 2005; Wunderlich 
et al., 2005). These results favor the notion that activ¬ 
ity in early visual areas may be important for aware¬ 
ness, as is suggested by the interactive model. However, 
scientists are still trying to discover whether the inter¬ 
active model or hierarchical model of awareness is a 
better description of the way the brain works. 

Another approach is to train monkeys to report 
which of two patterns is dominant during binocular 
rivalry (Figure 6.27). This takes some training but can 
be done by providing fruit juice rewards for the mon¬ 
key. Then, while the monkeys report their perception 
during rivalry, researchers can measure the activity 
of single neurons in different parts of the brain. These 
studies find strong modulations in the ventral tempo¬ 
ral cortex that match the monkey's reported percep¬ 
tion during rivalry (Sheinberg and Logothetis, 1997). 
In fact, around 85 per cent of the neurons recorded 
in the temporal lobe showed these modulations. 
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FIGURE 6.27 Monkeys and binocular rivalry. Illustration of 
monkey doing a perceptual binocular rivalry task, while research¬ 
ers record from neurons in the monkey's brain, as in Sheinberg and 
Logothetis, 1997. Source: Blake and Logothetis, 2002. 


Although related studies have found weaker effects of 
rivalry in the early visual areas of monkeys (Leopold 
and Logothetis, 1996), the studies described above 
reveal that it is indeed possible to study the neu¬ 
ral correlates of consciousness in both monkeys and 
humans. 


5.3 Visual detection: did you see it? 

Another way to separate physical stimulation and per¬ 
ceptual awareness is to get people to do a visual detection 
task. Here, a subject has to detect and say when he or 
she sees a particular pattern. The researcher makes 
the pattern harder and harder to see, so on different 
instances the pattern might be easy to see while at oth¬ 
ers times almost impossible and sometimes it won't 
even be there. Because this task gets difficult, peo¬ 
ple will get it wrong sometimes. There will be occa¬ 
sions when someone reports seeing the pattern when 
it wasn't even presented and other times when he or 
she will miss the pattern and report seeing nothing. 


When this kind of experiment is done in the fMRI 
scanner, we can see the pattern of brain activity corre¬ 
sponding to the visual pattern. Also, when no pattern 
is displayed and subjects report that they don't see the 
pattern (true negative ), we don't see that type of brain 
activity for the pattern. However, what do you think 
happens when someone gets it wrong? This is the case 
when no pattern is actually presented but the subject 
thinks he or she saw something and so reports 'yes the 
pattern was there' (false positive). Interestingly, activity 
in areas VI, V2, and V3 closely follows what we think 
we saw. In other words, on trials where a faint stimu¬ 
lus is presented but the subject fails to detect it, activ¬ 
ity is much weaker in these areas (Ress and Heeger, 
2003). However, if no stimulus is presented but the 
subject mistakenly thinks that a faint stimulus was 
presented, it turns out that activity is greater in VI, 
V2, and V3 on these trials. 

What accounts for these fluctuations in activ¬ 
ity level from trial to trial? This has been attributed 
to trial-to-trial variability in neural activity or neural 
noise in the system. Your memory of the pattern could 
'shape' the random neural activity present in sensory 
areas, tipping the balance and making you think you 
saw the pattern. This is another example of how the 
brain's activity may closely reflect the phenomenal 
experience of seeing something, even when nothing 
was actually presented. This is interesting because it 
demonstrates that it doesn't matter what physical pat¬ 
tern is presented to a person; what does matter is what 
is happening in the brain! 

5.4 Constructive perception: more to vision 
than meets the eye . . . 

If you drive a car then you probably know what a 
blind spot is - for the driver it's the area behind the car 
that cannot be seen in the side or rear view mirrors. 
Our eyes also have a blind spot , at the back of the ret¬ 
ina where the axons of the retinal ganglion cells meet 
to form the optic nerve as it exits the eye (see Figure 
6.3(b)). There are no photoreceptors in the blind spot 
and hence we are blind in that area. 

It is easy to demonstrate the blind spot. Look at the 
diagram in Figure 6.28(a). Close your left eye, look 
directly at the cross with your right eye, and move the 
textbook close to your nose. Then move it slowly away 
from your face, while keeping your eye fixed on the 
cross. At the right distance, which should be around 
12 inches (30 cm) away from the page, you should 
notice the dot vanish. As the image of the dot on your 
retina moves into the blind spot, it disappears! 
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FIGURE 6.28 Demonstrations of the blind spot, (a) Close your left eye, look directly at the cross with your right eye, move 
the page up close to your nose, and then move it slowly away from your face, while keeping your eye fixed on the cross. At the 
right distance, which should be around 12 inches (30 cm) away from the page, you should notice the red dot vanish. As the image 
of the dot on your retina moves into the blind spot, which has no photoreceptors, it disappears! (b) Likewise, notice how the black 
stripes now fill in; they become joined and the red dot vanishes. Source: Frank Tong, with permission. 


Hopefully this demonstration worked on you, and 
perhaps you are thinking .... wait a minute, if there 
is a blind spot in my vision all the time then why don't 
I see a hole in my vision when I look around at things 
with one eye covered? Well, the brain does something 
remarkable, it actually fills in perception of the blind 
spot. The brain uses the visual information from around 
the blind spot to infer what should be in the blind spot 
and it constructs awareness of what it 'thinks' should 
be there. Filling in at the blind spot is an example of 
constructive perception or perceptual filling in. 

Another way to demonstrate the filling in can be 
seen by viewing Figure 6.28(b). Move the page around 
until the gap between the lines is in the blind spot. 
When each group of lines is abutting the blind spot, 
what you will see are continuous long lines. The red 
dot should be replaced by one continuous set of lines. 
The brain fills in the path of the lines so you don't see 
the red dot in the gap anymore. This is a case of per¬ 
ceptual filling in; the brain actively creates the percep¬ 
tion of a complete object from the separate segments 
that lie on either side of the blind spot. 

Perceptual filling in not only happens in the blind 
spot, but also occurs in other parts of the visual field. 
Notice how the area between the colored lines in Figure 
6.29(a) somehow appears colored. However, careful 
inspection reveals that this background area is actually 
just white - no color at all. This illusion is called neon 
color spreading and the experience of color here seems to 
come from constructive filling-in mechanisms at work 
in the visual cortex. Recent brain imaging studies have 
found that activity in VI is greater in corresponding 
regions where subjects perceive neon color spreading 
(Sasaki and Watanabe, 2004). This suggests that neu¬ 
rons as early as VI may be important for perceptual fill¬ 
ing in and our experience of color, even illusory colors. 

During another type of filling in known as visual 
phantoms (Figure 6.29(b)), our visual system can also 
fill in gaps between two patterns. In Figure 6.29(b), you 
may have the impression of dark bands continuing 





(b) 


FIGURE 6.29 Demonstrations of perceptual filling in. (a) In 
these three examples the background is white - it has no color. 
However, you might notice that the red and the blue tend to fill 
in, coloring the white background, (b) The light patches and dark 
patches in the top and bottom panels tend to give the impression of 
light and dark sectors along the center strip, even though the center 
strip is a uniform gray. This illusion works much better when mov¬ 
ing. Source: Frank Tong, with permission. 


across the blank gap, even though there is no stimulus 
being presented there at all. Using moving stimuli can 
further enhance the visual phantom illusion. When we 
experience this type of filling in, cortical areas VI and 
V2 respond as if a real pattern were being presented 
in the gap (Meng et al., 2005). 

The brain can fill in not only color and patterns but 
also motion. If you flash a spot of light in one location. 
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then follow it by a similar flash at a different location 
at the appropriate time, observers experience the illu¬ 
sion of movement - apparent motion. This kind of trick 
is used all the time in overhead shop signs and it's 
the basis for why movies look so smooth and real. If 
you look at the actual film running through the pro¬ 
jector you will find that there is a series of stills that 
are flashed onto the screen one after another: the expe¬ 
rience of smooth motion is simply an illusion. Recent 
studies have found that the perceived path of apparent 
motion is represented by neural activity in VI (Jancke 
et al., 2004; Muckli et al., 2005). As with the previous 
examples of color and pattern, the brain seems to 
fill in apparent motion both perceptually and neurally, 
even when there is no physical motion occurring. 

From the above examples, it should be clear that 
the brain can actively construct perceptual represen¬ 
tations, even when there is no physical stimulus pre¬ 
sented in a particular region of the visual field. Filling 
in appears to occur in early visual areas, as early as 
VI, suggestive that early visual areas may provide the 
basis for constructive visual experiences. 

5.5 Neural correlates of object recognition 

What brain areas are important for object recognition? 
As we saw in the study of binocular rivalry, activity 
in the fusiform face area and parahippocampal place 
area is closely linked to the observer's awareness of 
faces and houses. Similarly, if subjects are presented 
with ambiguous figures such as those shown in Figure 
6.24(a), regions of the ventral temporal cortex show 
greater activity when recognition switches to the other 
interpretation. For example, when viewing Rubin's 
face-vase display, switches to the face percept lead to 
increases in activity in the fusiform face area. 

Other studies have investigated awareness of visu¬ 
ally masked objects, which are so briefly presented 
that they can just barely be recognized. When observ¬ 
ers are able successfully to recognize a briefly flashed 
object, activity is greater in many regions of the ven¬ 
tral temporal cortex than when recognition is unsuc¬ 
cessful (Grill-Spector et ah, 2000; Bar et ah, 2001). Also, 
if observers are shown an ambiguous black and white 
image, like a Mooney face shown in Figure 6.30, right 
panel, which when shown at different orientations is 
hard to recognize, activity is much greater in ventral 
temporal areas when subjects become aware of the 
hidden object (Dolan et ah, 1997; McKeeff and Tong, 
2006). Interestingly, in most of these studies, early vis¬ 
ual areas don't show a difference between conditions 
in which observers perceive the object or not. Perhaps 



FIGURE 6.30 These images are called Mooney faces. They are 
originally made from photos of real faces. Because these face images 
can be hard to recognize, they are useful in studying object recogni¬ 
tion. When people do finally recognize the face, you see an increase 
in neural activity in the fusiform face area. Source : Frank Tong, with 
permission. 

what is even more remarkable, when subjects are sim¬ 
ply asked to imagine a face, this can lead to activation 
of the fusiform face area (O'Craven and Kanwisher, 
2000). Likewise, imagining a place or landmark can 
activate the parahippocampal place area. These stud¬ 
ies suggest that simply retrieving a visual memory of 
a particular object can activate these object areas. 

6.0 MANIPULATIONS OF VISUAL 
AWARENESS 

In the previous section, we discussed correlational 
methods of studying conscious perception that relied 
on finding the neural correlates of consciousness. 
Knowing that activity in area A correlates with per¬ 
ception B (e.g. seeing leftward motion) provides corre¬ 
lational evidence that area A is involved in representing 
perception B. However, something can be correlated 
with a particular function without being causal. For 
example, the engine of a car roars whenever it is work¬ 
ing hard, but the roaring sound of the engine isn't 
what makes the wheels turn. Dampening the sound of 
the engine won't stop the car from moving, but dis¬ 
rupting the pistons from moving will. 

To provide further evidence of the role of a brain 
area in consciousness, we need to provide causal evi¬ 
dence showing that manipulating the activity in area 
A can produce or disrupt perception B. There are many 
causal methods in neuroscience, which involve induc¬ 
ing or disrupting activity in a certain brain area, either 
temporarily or permanently (e.g. lesion studies). This 
next section will highlight some of the findings from 
manipulations of neural activity and visual awareness. 
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Electrical stimulation of the occipital lobe can elicit 
subjective visual sensations or what are called phos- 
phenes. For example, blind patients who had electri¬ 
cal implants in primary visual cortex reported having 
their first visual experience in years when this area 
was stimulated. When small regions were stimulated, 
the subjects reported seeing phosphenes that looked 
like 'small grains of rice held at arm's length'. 

This seems obvious if we remember that the lan¬ 
guage of the brain is really electrical. Neurons commu¬ 
nicate via weak electrical currents, hence when a current 
is applied to a collection of neurons it is not surprising 
that the person experiences some sensation. There are 
many ways to do this: during a surgical operation neu¬ 
rons can be electrically stimulated or, on a larger scale, 
an electrical current can be passed through the skull. 
However, a more recent and less invasive method is 
transcranial magnetic stimulation (TMS). 

6.1 Transcranial magnetic stimulation 

Transcranial magnetic stimulation, or TMS, involves 
rapidly generating a magnetic field outside of the 
head to induce electrical activity on the cortical surface 
(Walsh and Cowey, 2000). This is usually done using a 
handheld TMS coil that can be placed on the surface 
of the scalp. When the electrical current is switched 
on the magnetic field is activated - triggering neural 
activity on the cortical surface (Figure 6.31(a)). This 
induced activity can either act as a type of 'visual 
stimulus' itself, or disrupt ongoing visual activity. 

6.1.1 Phosphenes and V1 stimulation 

When TMS is applied over early visual areas, there 
are two primary perceptual consequences. One, when 
people have their eyes closed they tend to experience 
a weak flash of light, a phosphene. As mentioned 
above, this is attributed to the activation of visual 
neurons. The second consequence is that you can 
experience a visual hole or momentary blind spot - 
a transient scotoma in visual stimulation just after the 
TMS (Kamitani and Shimojo, 1999). 

What is interesting is that the type of phosphene 
people experience corresponds to the area of cortical 
tissue stimulated with TMS. For example, when VI is 
stimulated, people report smallish static phosphenes. 
When area MT (the motion area) is stimulated indi¬ 
viduals report moving phosphenes (Pascual-Leone 
and Walsh, 2001)! 

These findings taken alone could be viewed as evi¬ 
dence that these areas are responsible for processing of 



FIGURE 6.31 Stimulating the brain, (a) A transcranial mag¬ 
netic stimulation (TMS) coil over the brain. The dashed lines show 
the magnetic field and the area tissue that is primarily affected. 
TMS is a valuable research tool to investigate the cause-effect rela¬ 
tionship between brain activity and visual awareness, (b) When a 
collection of these electrodes were stimulated the patient reported 
the experience of looking down at her own body - an "out of body 
experience'. Source : Blake and Logothetis, 2002. 

the corresponding visual features (objects, motion, etc.). 
However, we know that most parts of the brain are 
joined to most other parts and neural activity can 
travel very quickly from one location to another; 
hence, it might be the case that when you stimulate 
one area activity travels out to multiple other areas. 
So what if you used TMS stimulation at multiple loca¬ 
tions at different times? Might this inform us about 
neural transmission and neural feedback? 

6.1,2 TMS and cortical feedback 

A recent study provided new evidence to support 
the role of feedback connections in visual awareness 
(Pascual-Leone and Walsh, 2001). Motion phosphenes 
were reliably elicited by applying TMS to area MT/V5 
(motion areas) and a second TMS pulse was applied to 
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either VI or MT at various times before or after the first 
pulse. Perception of the motion phosphene was selec¬ 
tively impaired when VI stimulation occurred shortly 
after MT stimulation (lCMfims later), but not before¬ 
hand. In other words, researchers would first pulse MT, 
which is higher in the processing hierarchy. Then, at a 
given time later, they would pulse VI, which is earlier 
in the stream of processing. When this was done at the 
right time interval, subjects no longer perceived the 
phosphenes. This suggests that feedback projections 
from MT to VI might be necessary for conscious percep¬ 
tion of the phosphenes. The authors concluded that MT 
activity alone might be insufficient to support aware¬ 
ness of motion and that feedback activity to VI may be 
necessary for visual awareness. These findings provide 
support for the interactive theory of visual awareness. 

A TMS study of a blindsight patient, GY (remember 
that patients with blindsight typically have a large sec¬ 
tion of VI damaged), provides some evidence in favor 
of the feedback re-entrant model of visual awareness 
(Cowey and Walsh, 2000). In GY, stimulation of MT elic¬ 
ited motion phosphenes only when applied to the hemi¬ 
sphere housing the intact VI, but not on the side with 
damaged VI cortex. So even though area MT was per¬ 
fectly fine in both cortical hemispheres, stimulating MT 
on the same side of the head as the damaged VI did not 
lead to the perception of phosphenes. In contrast, motion 
phosphenes were successfully elicited in all normal sub¬ 
jects tested as well as in a retinally blind patient. Further 
tests of the interactive model should pursue whether 
direct cortical stimulation of extrastriate areas can elicit 
phosphenes in patients with VI damage. 

Another fascinating example of the modulation 
of visual awareness comes from a study in which 
researchers directly stimulated neurons in a monkey's 
brain (Salzman et al. r 1990). As we mentioned earlier, 
neurons in cortical area MT respond selectively to 
particular directions of motion and neurons that pre¬ 
fer similar directions tend to be clustered in cortical 
columns. In other words, a select bunch of neurons in 
MT might only be vigorously active when a monkey is 
presented with rightward motion. In this experiment, 
researchers directly stimulated some of these direc¬ 
tion-selective neurons in the monkey's area MT. They 
did this while the animal was viewing ambiguous dots 
moving in many possible directions, while performing 
a motion discrimination task. This microstimulation 
biased the animal's judgments toward the direction of 
motion that the stimulated neurons typically respond 
to. In other words, it was as if stimulating these neu¬ 
rons led to a stronger perception of that particu¬ 
lar motion direction, enough to shift the monkey's 


impression of motion a fair bit, if the motion in the 
stimulus was quite ambiguous. 

Much more complex perceptual experiences may be 
triggered by neural stimulation, even what might be 
called out-of-body experiences. An epilepsy patient who 
(due to the severity of her condition) had electrodes 
temporarily implanted in her brain reported such expe¬ 
riences. When two specific electrodes were stimulated 
over the temporal parietal junction (see Figure 6.31(b)), 
she reported the novel sensation of falling or float¬ 
ing (Blanke et al., 2002). She described the sensation 
as 'falling from a height'. Stronger stimulation led 
to a report of an apparent out-of-body experience. 
That's right, she claimed actually to experience see¬ 
ing her body from above, lying in bed. However, she 
did not see her own face, only from her trunk down¬ 
wards. This finding is important because the angle 
she reported seeing her body from was different from 
what she normally experienced. So, this suggests that 
familiar experiences (her normal body view) can be 
combined to form novel experiences of seeing her own 
body from different angles. This concept has exciting 
implications for the study of perceptual experience, 
because it suggests that experiences of an internal ori¬ 
gin do not simply have to be a repeat of previous per¬ 
ceptual phenomena. 

Stimulation studies allow scientists to pin down the 
cause-effect relationships between neural activity and 
perceptual experiences and can therefore reveal what 
brain areas are responsible for what we see. In essence, 
it is like taking control of the brain. By directly stimu¬ 
lating the cortex and bypassing sensory organs, such 
as the retina, it may be possible to identify which brain 
areas are more directly linked to perceptual experience. 

6.2 Unconscious perception 

How can something be both unconscious and still be 
called perception? Wouldn't the fact that you're not 
aware of something suggest that your brain was un¬ 
able to process or identify that stimulus? In the current 
context, we use the term unconscious perception to refer to 
situations when subjects report not seeing a given stim¬ 
ulus, but their behavior or brain activity suggests that 
specific information about the unperceived stimulus was 
indeed processed by the brain. Neural activity does not 
always lead to awareness. Neurons may fire in a stim¬ 
ulus-specific way at many levels of the brain, but this 
activity may not be strong enough, last long enough, or 
involve enough neurons or brain areas to lead to aware¬ 
ness. One of the best examples of this comes from neural 
recordings in animals under anesthesia; visual neurons 
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in many brain areas still show strong stimulus-selective 
responses. In this section, we will learn that even when 
a person is fully awake and alert, unconscious visual 
processing can also occur in many situations. 

In a previous section, you learned about how 'what 
you see is what you get activated' during binocu¬ 
lar rivalry. In other words, if you become conscious 
of a particular stimulus, then neurons in your brain 
that represent that pattern will become highly active. 
However, here we will learn that the opposite is not 
necessarily true. If you have some activity in a given 
brain area, this does not mean you will necessarily 
perceive the stimulus. Without enough activity in the 
right brain areas, awareness may simply fail: the result 
is unconscious perception. 

When two different stimuli are flashed briefly enough 
in quick succession, the visual system can no longer sep¬ 
arate the two stimuli. Instead, what people perceive is a 
mix, or a fused blend of the two images. For example, if 
we were to expose you to a red square, then quickly fol¬ 
low it with a green square, what you might experience is 
a combination of the two - a yellow square. This method 
can be used to present invisible patterns, such as a face 
or a house, by flashing red contours on a green back¬ 
ground to one eye and the opposite colors to the other 
eye (Figure 6.32). When subjects were presented with 
such images in the fMRI scanner, the fusiform face area 
responded more strongly to faces and the parahippocam¬ 
pal place area responded more to houses, even though 
subjects were not aware of the images (Moutoussis and 
Zeki, 2002). This is an example of unconscious percep¬ 
tion. The activity in ventral temporal parts of the brain 
could differentiate, to some degree, whether a face or 
house was presented, even though the subject could 
not verbally report a difference. So just because an area 
is somewhat active does not mean we will be aware of 
what that brain area represents. Perhaps this level of 
activity is just not enough. Stronger responses were 
found in these brain areas when just a single red-green 
pattern was shown so it was clearly visible. 

Area MT, a brain area specialized for processing 
motion, also responds to unperceived motion. When 
a moving stimulus is presented far out in the periph¬ 
ery of the visual field and is crowded by other stimuli, 
area MT still responds to motion - even when sub¬ 
jects are not aware of the motion (Moutoussis and 
Zeki, 2006). In this experiment, the motion stimulus 
was flanked on two sides by flickering stimuli, mak¬ 
ing the visual space so busy and cluttered that sub¬ 
jects could not see the motion in the display. Although 
MT responded somewhat to the perceived motion 
stimulus, it responded much more strongly when the 



FIGURE 6.32 Red + green = yellow. The red and green com¬ 
bine to form the yellow square of color. Hence, the images of the 
house and face become invisible when briefly shown one to each 
eye. However, the brain still responds to these unseen patterns. 
Source : Moutoussis and Zeki, 2002. 

surrounding flickering stimuli were removed and the 
motion stimulus was clearly perceived. 

In other studies, researchers used binocular rivalry 
(which we discussed in an earlier section) to render 
pictures of fearful faces invisible (Figure 6.33(a)). The 
expressions of the faces carried emotional content, 
although the subjects were never aware of seeing the 
faces. The amygdala, a brain region in the medial tem¬ 
poral lobe that normally responds to emotional stim¬ 
uli, also responded to the invisible fearful faces (Pasley 
et ah, 2004). Figure 6.33(b) is a plot of the activity in 
the amygdala; the red line represents the response of 
the amygdala to the emotional face images. 

It is clear that many brain areas may continue to 
show stimuli-specific activity despite the fact that 
we are unaware of that stimulus. One thing this sug¬ 
gests is that even if a brain area is processing a stimu¬ 
lus, this doesn't mean we will perceive that stimulus. If 
neural activity in a given area is not enough to result 
in awareness, then what is? In most of these studies, 
greater activity was found when subjects were aware of 
a stimulus than when they were not, suggesting that a 
minimum level of activity may be needed to make the 
difference between no awareness and awareness. A low 
level of neural activity below a specific threshold might 
not be adequate to result in being aware of a stimulus. 

This idea of a neural threshold for visual aware¬ 
ness fits nicely into both the hierarchical and interac¬ 
tive models of visual awareness we discussed earlier. 
For the hierarchical model, an adequate level of neu¬ 
ral activity would have to be maintained in high-level 
visual areas for awareness to occur. For the interac¬ 
tive model, feedback signals might act to boost neural 
activity at each level of processing, leading to activ¬ 
ity that surpasses the threshold for visual awareness 
in multiple brain areas. In sum, we can conclude that 
at least two things are needed for visual awareness: 
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FIGURE 6.33 The emotion is still perceived but not the face, (a) Schematic of the stimulus used in the Pasley et al. 
(2004) study. The red building presented to the left eye suppresses the face in the right eye out of awareness, as in bin¬ 
ocular rivalry. The faces have emotional expressions, (b) This graph shows the activity in the amygdala (an emotional 
response area of the brain). The red plot shows that the activity in the amygdala increases when emotional faces are pre¬ 
sented, even though they are out of awareness. The brain cannot see the face, but it can detect the emotion. Source : Pasley 
et al, 2004. 


(1) activity in the right neurons or brain areas and 

(2) activity that exceeds a critical threshold. Further 
research may show which of these models best 
describes the brain basis of visual awareness. 

7*0 SUMMARY 

Vision is perhaps our most important sense modality. It 
is certainly the one that has seen the most research. Over 
the past decade or so, scientists have learned a great deal 


about the neural correlates of conscious and uncon¬ 
scious perception and how the disruption of different 
brain areas can disrupt specific aspects of visual con¬ 
sciousness. A consistent finding is that primary visual 
cortex seems to be important for the ability to perceive 
any visual feature at all, while higher brain areas may 
be important for perceiving particular visual features 
or objects. Current evidence provides support for both 
hierarchical and interactive theories of visual awareness. 
Future studies will improve our understanding of how 
the brain gives rise to our subjective visual experiences. 
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In this chapter, we traced the functional properties 
of neurons as visual signals travel up from the retina to 
the primary visual cortex and onward to higher areas 
in the dorsal and ventral visual pathways. Progressing 
up the visual pathway, receptive fields gradually become 
larger and respond to more complex stimuli, following 
the hierarchical organization of the visual system. 

VI is selective for many visual features, including 
orientation, motion, and binocular disparity. Damage 
to VI can severely impair or eliminate conscious 
vision, although remaining activity in extrastriate 
areas may support the ability to detect visual events 
even without being visually conscious, the condi¬ 
tion called blindsight. Extrastriate visual areas (the 
ones outside of VI) seem to be important for perceiv¬ 
ing specific visual features: area V4 is important for 
color perception and area MT for motion perception. 
Damage to these areas may lead to selective impair¬ 
ment in the perception of these higher-level features 
of the visual world. 

According to the hierarchical theory, higher extra¬ 
striate areas are closely linked to visual awareness 


whereas VI is not. In contrast, the interactive theory 
emphasizes that feedback signals to VI may be impor¬ 
tant for awareness. Current evidence provides support 
for both theories. 

Damage to the dorsal pathway can lead to optic 
ataxia (impairments in visually guided actions) or visual 
neglect. Damage to the ventral temporal cortex can 
lead to impairments in visual perception, object recog¬ 
nition or face recognition. Patients with brain injuries 
in the ventral and dorsal pathways reveal a dissocia¬ 
tion between the conscious perception of basic shapes 
and orientations and the ability to perform visually 
guided actions. 

In the ventral temporal cortex, some brain regions, 
such as area LOC, seem to have a general role in object 
recognition, while other areas, such as the fusiform 
face area and parahippocampal place area, appear to 
have more specialized roles. Many studies show that 
activity in these areas is strongly associated with the 
conscious perception of objects. Nonetheless, evidence 
of unconscious processing can be found in many brain 
areas, including high-level object areas. 


8*0 STUDY QUESTIONS AND 
DRAWING EXERCISES 

1 For the brain drawing in Figure 6.34: 

a Copy the brain figure. Which way is it 
facing? 

b Identify the visual regions (labeled) and 
some of the things they do. 
c Can you color in the dorsal and ventral 
streams? What is the difference between their 
functions? 

2 In Figure 6.35, can you describe what is happen¬ 
ing in your own words? What is the cat seeing? 
Which visual areas are likely to be involved? 
What kind of neural mechanisms improve 
the ability of the cat to perceive contrasts and 
boundaries? 



Limit of 
visual cortex 


FIGURE 6.34 Visual areas of the human cortex. Source : 
Rosa, 2002. 
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3 For Figure 6.36: 

a Draw each figure in color, 
b What can we learn from (a)? What parts of 
the visual cortex are likely to be involved? 
c How about (b)? Are different parts of the 
visual cortex likely to be involved than 
in (a)? 

d For the third image in the figure, what does 
the subject in the experiment perceive? Why 
do the colors look mixed together? What is 
the purpose of this experiment and what are 
the results? 

4 A question to think about: how can we compare 
conscious and unconscious visual stimuli? 



FIGURE 6.35 Recording from the brain of a cat. Source : 
Rosa, 2002. 

















It was very fortunate that, even in Helmholtz’s time, the great anatomical discoveries by Cord 
(and others) had already made it clear that the vibrating tissue most important for hearing is the 
basilar membrane of the inner ear, because the cells on which the nerve endings terminate are 
seated on this membrane ... the problem of how we hear was reduced largely to a mechanical 
question: how does the basilar membrane vibrate when the eardrum is exposed to a sinusoidal 
sound pressure? 

Bekesy, Nobel Prize Lecture (online) 



The auditory system really starts deep inside the ear canal at the eardrum. Air vibrations, which are fast compressions and 
expansions of the air, vibrate the eardrum, which transmits mechanical vibrations that end up triggering receptors (hair 
cells) located on the basilar membrane. Axons from the receptor cells combine to make up the auditory nerve, which goes 
through several stages of processing before reaching cortex. The auditory system is extraordinarily sensitive, able to pick 
up even the sound of air molecules in very quiet environments. Sources: Top: Standring, 2005; bottom: Ramachandran, 2002. 
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1.0 INTRODUCTION 

This chapter provides an overview of how we hear - 
from simple sounds, to complex speech, to sympho¬ 
nies. We begin with basic information about how we 
process sounds: from the ear, through the ascending 
auditory pathways, to auditory cortex. Next, we dis¬ 
cuss specific types of sound processing such as speech 


and music perception. As you can imagine, sound 
perception changes throughout life, for example, as 
we acquire speech and language as infants and young 
children. You might have an intuition - and you 
would be correct! - that the neural systems underlying 
sound processing may be set up somewhat differently 
for people who are skilled musicians or for linguists 
who speak several languages fluently. Therefore, we 
will discuss the effects of learning and expertise on 
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brain systems for sound processing and how they dif¬ 
fer throughout life and across individuals. But sound 
processing does not happen in isolation - what we 
hear combines with what we see and touch, as well as 
with our stored memories and experiences. 

1.1 A model for sound processing 

Our environment is frequently quite noisy with many 
types of sounds reaching our ears at the same time. 
Think about a large college classroom before a lecture 
begins: there are the sounds of students' voices, chairs 
scraping, doors opening and closing, backpacks being 
unzipped, books being dropped onto desktops. All 
of these sounds hit our ears at the same time and yet 
we have little difficulty in perceiving them as sepa¬ 
rate events or auditory 'objects'. This process is called 
auditory scene analysis and forms the basis for under¬ 
standing how the auditory system decodes a complex 
listening environment (Bregman, 1990). In this chapter, 
we will discuss how the auditory system decodes this 
type of auditory scene. We will begin, however, with 
a functional framework with which to understand the 


processes of the auditory system and how they inter¬ 
act with other subcortical and cortical systems. 

1 . 1.1 A working framework for sound perception 

In Chapter 2, we discussed a modal model for under¬ 
standing brain processing. The same general concepts 
hold for auditory processing: sensory (sound) inputs 
enter the system and there is a very brief storage 
(echoic memory) for these inputs (Figure 7.1). Selective 
attention allows the system to direct its attention to 
a subset of the inputs for further processing. At this 
stage, there are complex interactions between the new 
inputs and existing memory and experiences, as well 
as with other sensory systems. The ultimate goal or 
'action' to be performed is important as well and will 
affect how information is encoded and stored. It is 
important to note that this model for auditory process¬ 
ing is not a one-way process, with sounds being 
decoded, understood, and then stored into long-term 
memory. There are interactions that occur throughout 
the encoding of sounds, both within the auditory sys¬ 
tem itself and across other sensory, cognitive, memory. 
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FIGURE 7.1 A functional framework for auditory processing, adapted from the general functional 
framework presented in Chapter 2. Sensory inputs, such as the sound of someone's voice or a cell phone ring, 
enter the system (see red arrows on the left side of the figure). There are early influences from bottom-up and 
top-down attentional processes (yellow arrows). These inputs make contact with working storage, long-term 
knowledge, and action systems. It is important to keep in mind that the processes underlying auditory func¬ 
tion are highly interactive, with feedforward, feedback, and integrative processes. 
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and motor systems. The anatomy and connectivity of 
the auditory system reflects this complexity, with mul¬ 
tiple stages of processing and neural pathways, includ¬ 
ing the ascending pathways from the ear to the brain, 
descending pathways that carry information back to 
the peripheral system, and many parallel pathways 
within brain regions and across the two hemispheres. 

1.1,2 Limited and large capacity 

As we discussed in Chapter 1, brain processes have 
both limited and large capacity aspects: this is the case 
for the auditory system. There are some specific limi¬ 
tations in decoding sound inputs. For example, if you 
present speech through headphones, it is easy to attend 
to each word uttered. However, if two different speech 
streams are presented to the two ears, it becomes a very 
difficult task to try to attend to each stream. In fact, we 
selectively listen to one stream or the other (Broadbent, 
1982). Thus, there are some limits to the capacity for 
decoding complex sounds entering the auditory system 
and a role for central executive function in directing 
attention selectively to some of the sounds in a complex 
listening environment. On the other hand, our capacity 
for learning new sounds or auditory objects (such as 
spoken words) continues throughout life and appears 
to be virtually unlimited in capacity. In fact, an aver¬ 
age adult's vocabulary is estimated at more than 
100,000 words. The same is true for recognizing new mel¬ 
odies and the voices of new friends and acquaintances. 
Therefore, while some capacity limits exist in attending 
to sounds during perception and encoding, once learned 
there appear to be virtually no limits regarding the capac¬ 
ity to remember new sound-based items. 


1.1.3 Orders of magnitude and levels of analysis 

As in other brain systems, auditory processing contains 
processing units that comprise many orders of magni¬ 
tude from individual hair cells at the periphery, to sin¬ 
gle neurons in auditory cortex, to large-scale neural 
networks in the auditory language system. The auditory 
system has been studied at each of these levels of anal¬ 
ysis in both human and in animal. In this chapter, we 
will include information that we have learned at each 
of these levels of analysis. However, a large focus of the 
evidence presented in this chapter will be on what we 
have learned about auditory processing at the system 
level from neuroimaging - positron emission tomog¬ 
raphy (PET), magnetic resonance imaging (MRI), func¬ 
tional MRI (fMRI), magnetoencephalography (MEG), 
and electroencephalography (EEG) - studies. The 


advent of non-invasive measures to investigate corti¬ 
cal processing has revolutionized the field of cognitive 
neuroscience and psychology in general. Previously, 
we relied on data from animal studies, made infer¬ 
ences from behavioral and psychophysical studies with 
healthy individuals, or investigated sound and language 
processing in humans who had suffered brain damage 
due to injury, disease, or stroke. The capability of inves¬ 
tigating brain processes in healthy individuals has pro¬ 
vided us with a wealth of new information about sound 
and language processing. It has also provided us with 
the ability to investigate brainwide processes in large- 
scale systems that span multiple brain regions, such as 
the language system. 

1.1.4 Time 

Time is a critical aspect of auditory processing: the audi¬ 
tory system differs from the visual system in that all 
sound processing occurs over time. Nothing 'stands 
still' in sound processing. Speech, the most complex 
signal that the auditory system must decode, has dif¬ 
ferences in speech sounds (phonemes) such as /b/ and 
/p/ that occur on a scale of 20-30 thousandths of a sec¬ 
ond (milliseconds), and yet our speech perceptual pro¬ 
cesses decode these transient differences with ease, even 
in relatively noisy environments (Gage et al., 1998,2002). 

Thus, the speech decoding system has a high tempo¬ 
ral resolution of fine-grained and transient changes at 
the level of the phoneme. However, the speech system 
also needs to decode information that changes over a 
longer time span than those contained within phonemes: 
syllabic stress (such as the different pronunciation of 
'melody' and 'melodic') is an important speech cue and 
occurs in a time window of approximately 200ms. Other 
key information occurs over 1-2 seconds (1000-2000 ms) 
at the level of a sentence, such as the rising intonation 
that is associated with asking a question. Thus, each of 
these time windows - 20, 200, 2000 ms - is critical to the 
accurate decoding of speech and information extracted 
from each of these decoding processes must be available 
for integration in the complex processes underlying the 
mapping of sound onto meaning (Figure 7.2). 

Before we begin our discussion of how the brain 
processes complicated listening environments, with 
human voices, complex environmental sounds, music, 
we need to discuss some basic principles of sound and 
hearing. We will begin with the physical features of 
sounds and how these features correspond to psycho¬ 
logical aspects of sounds. Next, we will step through 
the processes and stages of peripheral hearing and 
subcortical feature extraction. 
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1.2 Sound and hearing basics 

1.2.1 Physical features of sounds 

How does the human auditory system transform 
sounds into comprehensible speech or recognizable 
melodies? Let's begin with how we encode simple 
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FIGURE 7.2 A spectrogram is a picture of the sound-based 
features in speech. Time is represented on the x-axis and frequency 
is represented on the y-axis. The darker shading represents higher 
intensity. Speech contains harmonic content (formants) at specific 
regions in the spectral (frequency based) aspect of the spectrogram. 
Here we show a spectrogram showing three time scales critical for 
decoding speech. Upper left : Detail of the transients at the onset of 
a consonant, with transitions that occur on a time scale of —20 ms. 
Upper right: Detail of the formants in a syllable which occurs on a 
time of —200 ms. Bottom: A sentence that occurs on a time scale of 
—2000 ms. 


sounds at the level of the ear. A physical definition 
of sound is the vibration that occurs when an object 
moves in space, producing an audible sound. What we 
hear is not the vibration itself but the effects of vibra¬ 
tion in sound waves that move, or propagate, through 
space and make contact with our ears. The sinusoid 
(Figure 7.3) is a basic building block of sound that has 
three main physical aspects: frequency, intensity and 
time. The frequency of a sound is the rate of sound 
wave vibration and is measured as cycles completed 
per second, or hertz (Hz). A sinusoid with 1000 cycles 
per second has the frequency of 1000 Hz. The human 
auditory system can detect sounds across a wide 
range of frequencies, estimated at 20 to 20,000 Hz. 

The intensity of a sinusoid reflects the ampli¬ 
tude (or displacement) of the wave within its cycle 
and over time. In Figure 7.3 (left panel), we show a 
1000 Hz sinusoidal tone in the time domain, with time 
on the x-axis and intensity on the y-axis. On the right 
panel of Figure 7.3, we show the same tone in the fre¬ 
quency domain, with the frequency (Hz, or cycles per 
second) on the y-axis and time on the x-axis. Note that 
the spectral energy of a sinusoidal tone is limited to a 
single narrow band, so a 1000 Hz tone has energy cen¬ 
tered only at 1000 Hz. This is why a sinusoidal tone is 
frequently referred to as a 'pure' tone. 

Of course, most sounds that we hear are more com¬ 
plex than a pure tone. A piano chord, a car horn honk¬ 
ing, a person's voice, all have complicated structures. 
How do we describe these complex sounds in terms of 
the three physical parameters of frequency, intensity, 
and time? Joseph Fourier (1768-1830), a Frenchman 
who lived in the Napoleon I era, developed a series of 
theorems that describe how even complex signals can 
be separated into a series of simpler constituent parts 



Time (msec) 


FIGURE 7.3 Left panel shows a sinusoidal tone. Time is represented on the x-axis, amplitude is shown 
on the y-axis. The frequency of the sinusoid is based upon the number of cycles per second; thus a 1000 Hz 
tone has 1000 cycles per second. Right panel shows the same sinusoidal tone with its frequency focused on 
1000 Hz. Sinusoidal tones have a single frequency, which is why they are referred to as "pure' tones. Most 
sounds we hear are spread over multiple frequency bands. Sources: Left: Brown, 2003; right: Boatman, 2006. 
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through what is now called a Fourier analysis (Fourier, 
1822). The work of Fourier was advanced by Georg 
Ohm (1789-1854), who proposed that the separation 
of complex sounds into simpler sinusoids occurred at 
the ear in hearing. 

While we have mentioned the frequency, intensity, 
and time of a sound as comprising the basic physi¬ 
cal features, sounds have other qualitative aspects. 
For example, if you heard someone play middle C 
(—261 Hz) on a piano while at the same time an oboist 
played middle C, could you tell these sounds apart in 
spite of the fact that they are of identical frequency? 
Of course you could easily do so, suggesting that there 
must be many more dimensions in sound quality than 
just frequency. In this example, the timbre or quality of 
the note helps us distinguish between musical instru¬ 
ments, even when the notes they produce are identi¬ 
cal in frequency. Timbre also allows us to distinguish 
human voices. 


1.2.2 A scale for sound intensity 

The dynamic range of the human hearing system is 
extremely broad: we can hear barely perceptible sounds 
of very low intensity and very loud sounds that actually 


cause pain. This range has been calculated as ranging 
from 1 unit of intensity to 1,000,000,000,000,000 (10 15 ) 
units. This range is so large that it is difficult to deal 
with using normal numbering schemes. We typically 
use a logarithmic scale in order to deal more easily 
with the huge range in units of intensity, the decibel 
(dB) system. The dB scale is a relative (not absolute) 
scale and is based upon the ratio of two quantities: the 
relative intensity of a sound based on either the sound 
pressure level (SPL) in the air where hearing is occur¬ 
ring, or based upon the hearing threshold or sensation 
level (SL) of an individual. (Note: there are many other 
ratios used in describing hearing. We use SPL and SL 
here because they are common ratios used to describe 
sound intensity.) Human hearing ranges from — 1 
(threshold) to 150 dB SPL (Figure 7.4). 


1 .2.3 Psychological aspects of sounds 

While sounds have physical parameters (frequency, 
intensity, time) that can be measured with a fine degree 
of accuracy, how do we know how they are perceived? 
The physical parameter of frequency, or cycles per sec¬ 
ond, corresponds to the psychological or perceptual 
quality of pitch. Pitch is a subjective perception, usually 



FIGURE 7.4 Hearing threshold and range of hearing for human listeners. Shown also 
are the ranges of frequency and sound pressure levels of common environmental sounds, 
including human speech. The most intense sounds are capable of damaging the inner ear 
receptor organ. The hearing sensitivity of the cat, a laboratory animal commonly used in 
studies of the peripheral and central auditory system, is illustrated as well. Source : Adapted 
with permission from Brugge and Howard, 2002. 
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FIGURE 7.5 Drawing of the auditory periphery within the human head. The external ear (pinna and exter¬ 
nal auditory canal) and the middle ear (tympanic membrane or eardrum, and the three middle ear ossicles: mal¬ 
leus, incus, and stapes) are indicated. Also shown is the inner ear, which includes the cochlea of the auditory 
system and the semicircular canals of the vestibular system. There are two cochlear windows: oval and round. The 
oval window is the window through which the stapes conveys sound vibrations to the inner ear fluids. Source : 
Brown, 2003. 


described as the 'highness' or 'lowness' of a sound, for 
example, the pitch of a person's voice or of a note on a 
piano. We use the physical and psychological terms dif¬ 
ferently when discussing sound perception. Here's why: 
while we may know the frequency of a sound because 
we have measured the cycles per second, we do not 
know the precise pitch that an individual experiences. 
A highly trained opera singer, for example, may have a 
very different sense of the differences in pitch between 
closely matched sounds than an untrained individual, 
even though both have normal hearing. This is also 
the case for the physical parameter of intensity, which 
corresponds to the subjective perception of loudness. 
Individual listeners have a wide variety in how they 
perceive the loudness of sounds, depending on many 
factors ranging from hearing loss to personal prefer¬ 
ence. Therefore, it is important when describing sounds 


to be aware if you are describing the measured physical 
parameters or the subjective psychological features. 

1.2 A From the eardrum to the auditory nerve 

As we mentioned above, we will step through the 
stages of hearing and subcortical feature extraction 
processes as sounds are transmitted to auditory cor¬ 
tex. You will see that there are complex mechanisms 
at work, even in decoding relatively simple sounds 
in a quiet environment. Let's begin with how sounds 
proceed from vibrations at the eardrum, through the 
fluid of the inner ear, to innervate fibers at the audi¬ 
tory brainstem on their way to auditory cortex. 

Vibrating objects cause sound waves to move 
through air. When these sound waves reach the tym¬ 
panic membrane, or eardrum, they propagate through 
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FIGURE 7.6 Upper panel depicts the transmission of sound, with a perspective view of the cochlea showing the basilar 
membrane. Note that the red arrows depict sound transmission and are bidirectional. Lower panel depicts a traveling wave of 
sound as it crosses the basilar membrane. The wave is shown as frozen in time and somewhat exaggerated in order to illus¬ 
trate the movement across the basilar membrane by sound. Sources : Top : Drake, 2005; bottom: Javel, 2003. 


the middle ear through the mechanical action of the 
three bones of the middle ear: the hammer, anvil, and 
stirrup, to the cochlea, the organ of hearing in the 
inner ear (Figure 7.5) (for more on hearing fundamen¬ 
tals, see Moore, 1995). 

At the stage of the cochlea, in the inner ear, the phys¬ 
ical aspects of the sounds are encoded. (See Figure 7.6, 
upper panel, for a perspective of the cochlea showing 
the shape of the basilar membrane.) The traveling wave 
of sound moves across the basilar membrane from 
the base to the apex (Figure 7.6, lower panel). The 
basilar membrane is topographically organized in 
a frequency-specific manner, called tonotopy , with 
higher frequencies encoded at the base and lower 
frequencies encoded at the apex. 

How is the traveling wave converted to a neural 
code and transmitted to the brain? Within the cochlea, 
there are approximately 16,000 sensory receptors called 
the hair cells. The motion of the traveling wave along the 


basilar membrane sets the tiny hair cells into motion. 
The peak amplitude of the traveling wave causes maxi¬ 
mal bending of the hair cells located in specific regions 
or places of the basilar membrane, thus encoding the 
frequency of sounds. This is called the place principle 
of hearing and is based on the theory that the brain 
decodes the frequencies heard based upon which hair 
cells along the basilar membrane are activated. 

At this stage of processing, the movement of the 
hair cells produced by the traveling wave of sound is 
transformed or transduced into electrical responses in 
fibers of the auditory nerve (Kelly et al., 1996). Specific 
hair cells map onto to specific fibers in the auditory 
nerve and these fibers have a characteristic frequency to 
which they are most sensitive. 

The fine mapping of hair cells onto auditory nerve 
fibers preserves the frequency information in the 
sound as it is converted from vibration at the eardrum, 
to mechanical movement in the middle ear, to traveling 
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FIGURE 7.7 Top panel depicts innervation patterns of afferent and efferent neurons in the organ of Corti, located within the cochlea (see 
Figure 7.6). Afferent innervation is provided by ganglion cells of the spiral ganglion in the cochlea, which have central axons that form the audi¬ 
tory nerve. There are two types of afferent neurons: (1) type I neurons, which receive synapses from inner hair cells, and (2) type II neurons, which 
receive synapses from outer hair cells. The central axons of these ganglion cells form the auditory nerve. Efferent innervation is provided by a 
subgroup of neurons in the superior olivary complex that send axons to the cochlea and are hence called the olivocochlear (OC) neurons. There 
are two types of OC neurons: (1) lateral OC neurons, which innervate type I dendrites near inner hair cells, and (2) medial OC neurons, which 
innervate outer hair cells. Lateral OC neurons are distributed mainly ipsilateral to the innervated cochlea, whereas medial OC neurons are distrib¬ 
uted bilaterally to the innervated cochlea, with approximately two-thirds from the contralateral (not illustrated) and one-third from the ipsilateral 
side of the brain. Bottom panel depicts auditory nerve fiber tuning curves, with a fiber tuned to 2000 Hz (left curve) and 15 000 Hz (right curve); 
frequency is shown on the x-axis. The y-axis is threshold in decibels sound pressure level. Sources: Top: Brown, 2003; bottom: Javel, 2003. 


wave at the cochlea, to neural coding at the auditory 
nerve. A schematic of the pathways at the auditory 
brainstem is shown in Figure 7.7, top panel, with fibers 
that show a 'best' or center frequency response in the 
bottom panel. 

Thus, in this manner, sounds travel from the exter¬ 
nal ear canal, through the middle and inner ears, and 
on to the auditory brainstem. At this stage, the infor¬ 
mation in sounds is transduced through fibers on its 
way to auditory cortex. You may have noticed that long 
before the information in sound reaches cortex, there 
are many recodings and transforms that occur. These 
transformations of sound inputs are key to understand¬ 
ing how the brain decodes the multiple complexities of 
an everyday listening environment, such as the college 
classroom example described early in this chapter. 


2.0 THE CENTRAL AUDITORY SYSTEM 

The information in sound undergoes many transfor¬ 
mations as it ascends to auditory cortex. In this sec¬ 
tion, we will review the major aspects of the anatomy 
and neurophysiology of the central auditory system. 
The auditory system is comprised of many stages and 
pathways that range from the ear, to the brainstem, to 
subcortical nuclei, and to cortex. The three main divi¬ 
sions of the auditory system are the peripheral sys¬ 
tem, which we have already discussed, the pathways 
(ascending to cortex, descending from cortex, and 
parallel pathways across cortical sites), and the central 
(cortical) system. While each stage and pathway has 
functional significance in the decoding of sounds, it is 
important to consider the auditory system as a whole 
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FIGURE 7.8 Illustration of the human auditory system showing pathways and subcortical nuclei in 
the ascending and descending pathways. Source : Standring, 2005. 


because of the complex interactions across and within 
its constituent parts. 

2.1 Auditory pathways 

As we mentioned earlier, all sound processing occurs 
over time. The hallmark of the auditory system is its 
exquisite temporal resolution for decoding intricate 
information in sounds (Gage and Roberts, 2000; Gage 
et ad., 2006). One important aspect of the high temporal 
resolution of the auditory system is the fast and accurate 
transmission of sound information along and through¬ 
out its many pathways. Not only do transient features 
in complex sounds - such as the harmonic structure of 
consonants in speech or musical phrases - need to be 
conveyed rapidly from eardrum to cortex, but the infor¬ 
mation from the two ears needs to combined and inte¬ 
grated in a meaningful way en route. Let's discuss how 
and where this happens in sound processing. 


The ascending (afferent) pathways transmit informa¬ 
tion about sounds from the periphery to cortex. There 
are many stages of computation along the way: this 
pathway is not a simple delivery system but entails a 
significant amount of encoding and recoding of infor¬ 
mation in the sounds. The neural signal travels from the 
auditory nerve to the lower (ventral) cochlear nucleus. 
The cochlear nucleus is tonotopically organized. From 
the cochlear nucleus, the signal continues along the 
ascending pathway through the lateral lemniscus, infe¬ 
rior colliculus, thalamus, to auditory cortex (Figure 7.8). 
This is not a single pathway, but is complex and includes 
many computational stages as well as the combination 
of sound inputs from the two ears. A key function of the 
ascending pathway is to evaluate the information from 
the two ears in order to localize sounds in space - and 
we will discuss this in more depth later in the chapter. 

The descending (efferent) pathways from regions in 
the cortical and subcortical auditory system cortex to 
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the periphery are under direct or indirect cortical con¬ 
trol. Recent research indicates that this control extends 
all the way to the hair cells in the cochlea! One impor¬ 
tant function of the descending pathway is to provide 
'top down' information that aids in selective attention 
processes and in perceiving sounds in a noisy envi¬ 
ronment. The precise way in which the descending 
pathways function in sound processing is not well 
understood in humans. However, here is an example 
of some aspects of listening in which the descending 
pathways play a role. Imagine that you are having 
a very important conversation with a close friend as 
you stand outside a college classroom. You are focus¬ 
ing on the conversation, but meanwhile a motorcycle 
starts up in an adjacent parking lot, a helicopter passes 
overhead, and a gardener mows the nearby lawn with 
a power mower. You struggle to hear your friend, but 
manage to tune out most of these competing sounds. 
Suddenly the doors of the classroom open and scores 
of students erupt from the classroom, chatting and 
laughing. This may put an end to your conversation 
for a few moments, however, throughout the process of 
listening to your friend's voice during this noisy scene, 
your auditory pathways have been at work helping 
you both to focus your attention specifically to your 
friend and in extracting your friend's voice out of the 
competing noises coming at you from all directions. 

The auditory pathways are not just ascending to or 
descending from cortex, there are many important con¬ 
nections between the auditory cortices in the left and 
right hemispheres via the corpus callosum. These con¬ 
nections between the hemispheres are tonotopically 
organized. There are also cortico-cortical pathways that 
provide integration of auditory processes with other 
sensory systems, as well as with working and long-term 
memory processes, and stored memories and knowl¬ 
edge. Together with the ascending and descending 
pathways, the cortical pathways represent complex con¬ 
nectivity patterns that are critical, not only for process¬ 
ing sound, but also for integrating information to other 
regions in the brain. 

2.2 Auditory cortex 

At last we have arrived at auditory cortex and, in this 
section, we will discuss the anatomy of brain areas in 
auditory cortex as well as the neurophysiological fea¬ 
tures of these regions of cortex. Auditory cortex is the 
region within cortex specialized for sound processing. 
It is located in each hemisphere within the Sylvian fis¬ 
sure on the surface of the supratemporal plane and the 
upper banks of the superior temporal gyrus (Figure 7.9). 


As we have discussed, information in sounds is 
transmitted from the ear to auditory cortex via the 
ascending auditory pathways. Along the way, the sig¬ 
nal is transformed and recomputed in many ways. 
Auditory cortex is not the end stage of this pathway, 
but serves as a hub or nexus for sound processing, 
interacting dynamically with other systems within 
cortex, across the hemispheres, and back down the 
descending pathways to the cochlea. These dynamic 
processes provide a wide range of perceptual acu¬ 
ity and allow us to perform complex perceptual tasks 
such as selectively listen to one person's voice in a 
crowded and noisy room or recognize a melody even 
though it is played in another key or at a different 
tempo. These perceptual tasks are so complex that we 
do not totally understand how the auditory system 
performs them. However, we do them every day with 
little or no conscious effort on our part. 

Auditory cortex is not a unitary brain area but is 
comprised of several structural (anatomical) areas that 
differ in their role in decoding sound. Early descrip¬ 
tions of these areas within auditory cortex were made 
based on the structure, such as a gyrus within this 
cortical region, and by underlying neurophysiologi¬ 
cal features, such as the cytoarchitectonic classifica¬ 
tion. Although we do not fully understand the role of 
each area within human auditory cortex, work is in 
progress to map out regions within auditory cortex 
and their corresponding role in perception. We discuss 
our current knowledge about human auditory cortex 
below, with a description of the structure or anatomy 
followed by details regarding the cellular organization 
and response properties, or neurophysiology. 

2,2,1 Auditory cortical anatomy 

Much of what we know about auditory cortical 
anatomy comes from work in non-human primates 
(Galaburda and Pandya, 1983). In macaque monkeys, 
the major regions of auditory cortex are the core, belt, 
and parabelt regions (Figure 7.10, left panel). These 
distinct regions are distinguished by their cytoarchitec- 
tural, physiological, and connective properties. The core 
receives inputs from the thalamus and in turn projects 
to regions in the lateral belt areas (Kaas et al., 1999). This 
anatomical and connective structure has led scientists to 
suggest that there is a hierarchical organization in audi¬ 
tory cortex, with subcortical projections to the core or 
primary auditory (Al) region. Al is thought to decode 
basic properties or features in sounds. From Al, there 
are projections to the belt regions where more complex 
feature extraction processes occur (Kaas et al., 1999). 
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FIGURE 7.9 Top panel shows an illustration of the human brain from a lat¬ 
eral view, bottom panel from a medial view. Colored brain regions are adapted from 
Brodmann (1909). Auditory and receptive language cortical regions include Brodmann 
22,41,42, and 52. 
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FIGURE 7.10 Left panel depicts levels and regions of auditory cortical processing in the macaque monkey The areas shown are located 
on the superior temporal gyrus and in the depths of the superior temporal sulcus, which has been opened to show the extension of auditory- 
related cortex into this sulcus. Right panel depicts similar regions in the human, with the Sylvian fissure opened to show regions of auditory 
cortex on the supratemporal plane. (Adapted with permission from Zatorre, 2002.) Source : Caplan and Gould, 2003. 
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FIGURE 7.11 Left panel illustrates regions within auditory cortex, showing typical pattern of asymmetries in the left and 
right hemispheres. Right panel shows an MRI depiction of localizations of sound processing in the same regions. Sources: Left : 
Standring, 2003; right : Frackowiak, 2004. 


The belt and parabelt regions surrounding A1 are spe¬ 
cialized for sound processing and thus are unimodal 
auditory cortex. Human auditory cortex (see Figure 
7.10, right panel) shows similar regions, with the 
Sylvian fissure opened to show regions of auditory 
cortex that correspond to the regions presented for the 
macaque in Figure 7.10's left panel. 

In humans, primary auditory cortex is located within 
Heschl's gyrus (Figure 7.11, left panel) and is roughly 
analogous to core regions described in non-human 
primates. Heschl's gyrus corresponds to Brodmann's 
area 41 (Brodmann, 1909). Typically, primary auditory 
cortex comprises only a portion (one- to two-thirds) 
of the medial aspect of Heschl's gyrus. There is sig¬ 
nificant variability in the anatomy of Heschl's gyrus 
both in the two hemispheres and across individuals: 
Heschl's gyrus is typically located somewhat ante¬ 
rior (~6mm) in the right hemisphere than in the left, 
and some individuals have more than one Heschl's 
gyrus. This structural variability in Heschl's gyrus has 
important implications when interpreting functional 
neuroimaging findings, since the actual size and loca¬ 
tion of Heschl's gyrus vary so much across individuals 
(Figure 7.11, right panel; Frakowiak, 2004). 

Auditory cortex extends from Heschl's gyrus in the 
anterior-inferior direction and the posterior-superior 
direction along the supratemporal plane and the 
upper bank of the superior temporal gyrus. A second 
important anatomical region in human auditory cor¬ 
tex is the planum temporale , located just posterior to 
Heschl's gyrus. There are both hemispheric and indi¬ 
vidual differences in the planum temporale. How¬ 
ever, unlike Heschl's gyrus, the differences fall into 


a general pattern: the planum temporale is typically 
much larger in the left hemisphere than in the right. 
In fact, the left planum temporale can be up to ten 
times larger in the left hemisphere in right-handed 
individuals (Figure 7.11, left panel). Planum tempo¬ 
rale asymmetries were reported in a series of anatomi¬ 
cal studies by Geschwind and colleagues (Geschwind 
and Galaburda, 1985a, b, c). These scientists noted that 
language function tends to be lateralized to the left 
hemisphere. They suggested that the larger left hemi¬ 
sphere planum temporale reflects its role in decoding 
auditory language. More recent neuroimaging stud¬ 
ies, however, have provided a method to specifically 
test this hypothesis and provide differing views of the 
role of the planum temporale in sound processing. 
Just anterior to Heschl's gyrus is the planum polare. 
This region has not been the focus of much study in 
humans, and little is known about its role in audi¬ 
tory perception. Posterior to the planum temporale 
and unimodal auditory areas is Brodmann area 22. 
This is the area that Carl Wernicke hypothesized 
played an important role in speech comprehension 
(Wernicke, 1874/1977). According to Wernicke, this 
region was not an auditory region per se, but formed 
the language area for speech comprehension processes 
that were closely related (physically and functionally) 
to auditory processes. This region is typically referred 
to as Wernicke's area. 

2.2.2 Neurophysiology 

Several guiding principles of auditory cortical organi¬ 
zation have been established in studies of cats and 
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FIGURE 7.12 Schematic of the main anatomical cell types of the cochlear nucleus and their corresponding 
post-stimulus time (PST) histograms. Left : An auditory nerve fiber is shown with its typical response, a primary-like 
PST. Center. The auditory nerve fiber divides to innervate the main cochlear nucleus cell types. Right : PST histo¬ 
grams corresponding to these cell types are shown. In their PST histograms, pauser units fire an initial spike and 
then have a distinct pause before a slow resumption of activity Onset units fire mainly at the tone burst onsets. 
Primary-like units get their name from the similarity of their PSTs to those of primary auditory nerve fibers, but the 
primary-like with notch type additionally has a brief notch following the initial spike. Chopper units have regular 
interspike intervals that result in regular peaks in the PST. Most of these patterns are very different from the pri- 
mary-like PST and irregular interspike intervals of the auditory nerve fiber. For histograms, the sound stimulus is 
typically a 25 ms tone burst with frequency at the center frequency of the neuron and sound level at 30 dB above 
threshold. Source : Brown, 2003. 


non-human primates. The basic units of organization 
in auditory cortex, as in other cortical sensory areas, 
are neurons, cortical columns, and neural networks. 
There are several differing types of neurons in the 
auditory system (Figure 7.12). These neurons have dif¬ 
ferent response properties for coding frequency, inten¬ 
sity, and timing information in sounds as well as for 
encoding spatial information in processes for localiz¬ 
ing sounds in space. Most cortical neurons respond to 
binaural inputs (inputs from both ears), demonstrat¬ 
ing the importance of cortical processes for decod¬ 
ing binaural information for sound localization and 
other complex hearing processes. Together, these dif¬ 
fering types of neurons form a dynamic network that 
encodes transient features in sounds. Far less is known 
about the types of neurons in human auditory cortex. 
However, the large pyramidal cells that are distributed 


along the supratemporal plane likely play a key role 
in sound processing. 

Mapping receptive field properties of neurons in audi¬ 
tory cortex has been the focus of many animal studies. 
A large proportion of auditory cortical neurons have 
inputs from both ears. However, the two ears are not 
represented in the same way within each hemisphere. 
In auditory cortex in the left hemisphere, the right 
ear, the contralateral ear, has a much larger or stronger 
representation than the left ear, the ipsilateral ear. 
A similar and opposite pattern is observed in the right 
auditory cortex, with a stronger representation of the 
left ear versus the right. This general asymmetry for 
the representation of the two ears in binaural hearing 
holds true for humans as well (Woldorff et al., 1999). 

One important aspect of decoding features in 
sounds is determining where the sound is in relation 
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FIGURE 7.13 Receptive fields of two auditory cortex neu¬ 
rons plotted as a function of sound pressure level and azimuth in 
the frontal hemifield. Noise bursts were used as stimuli. Small dia¬ 
monds show points of maximal response, and progressively lighter 
shading shows regions of progressively smaller response. Zero 
degrees azimuth refers to directly ahead, and positive azimuths 
refer to points in the contralateral hemifield. Source : Brown, 2003. 


to the listener. In Figure 7.13, we show receptive field 
properties of two auditory cortical neurons as a func¬ 
tion of where the sound is located in space relative 
to the head (the x-axis), and of how loud the sound 
is (the y-axis). The receptive field of the neuron pre¬ 
sented in the top panel shows a large shaded region 
across sounds coming from the contralateral ear and 
across a lot of sound intensities. These receptive field 
properties indicate that this neuron is broadly tuned 
to sound information across loudness levels from the 
contralateral ear. In the lower panel, the neuron shows 
a sensitivity that is much more narrowly focused, in 
this case to sounds that are presented at 30-60 dB and 
coming from the ipsilateral ear. This neuron is nar¬ 
rowly tuned. Broadly and narrowly tuned neurons play 
differing roles in sound processing. A broadly tuned 


neuron may not provide detailed information regard¬ 
ing just where a sound is located in space or precisely 
how loud it is. However, this neuron will be highly 
sensitive to detecting any sound within a large loud¬ 
ness scale coming from the contralateral ear. Thus, this 
neuron may be important for the detection of the pres¬ 
ence of a new sound and provides general information 
about which ear the sound is coming from. A nar¬ 
rowly tuned neuron will provide more specific infor¬ 
mation regarding where a sound is, not just which ear 
but where in space on that side of the head, as well as 
more specific information about how loud the sound 
is. Thus, this neuron may be important for the discrim¬ 
ination of finer details about a sound. 

As in other cortical sensory regions, auditory cortex 
has a columnar organization, meaning that neurons are 
organized into columns that span across all six cortical 
layers (see Chapter 3). Within an individual column, 
neurons show similar response properties. A general 
scheme for the columnar organization in auditory cor¬ 
tex is that neurons that respond to binaural inputs are 
organized into alternating columns that have differ¬ 
ing roles in sound processing, either causing an exci¬ 
tatory (summation) or inhibitory (suppression) effect 
on information coming from the two ears (Brugge and 
Merzenich, 1973). These complex interactions of sum¬ 
ming or suppressing the information coming from the 
two ears likely underlie perceptual functions, such 
as selectively attending to information coming from 
one ear. 

A central guiding principle for non-human auditory 
cortex is the tonotopic organization. Within the core in 
cat auditory cortex, for example, receptive fields of 
neurons reflect a tonotopic organization in primary 
(Al) regions that has a mirror image in adjacent (ante¬ 
rior and posterior) core regions (Figure 7.14). 

While the basic aspects of neurophysiology are 
likely similar for humans, the uniqueness of human 
speech, language, and music perception, as well as the 
substantially larger regions of cortex devoted to audi¬ 
tory cortex in humans, probably mean that there are 
neurons and networks that are specialized for these 
complex processes and specific to human auditory 
cortex. For example, although many auditory cor¬ 
tical regions in non-human primates reflect a tono¬ 
topic organization, evidence for tonotopy has been 
less robust in human studies and may be limited to 
primary auditory cortex and not represent the basic 
organizational principle in non-primary auditory 
areas (Wessinger et al., 2001). The investigation of the 
organizational principles of human auditory cortex is 
a work in progress, much aided by the advent of new 
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FIGURE 7.14 Auditory cortical fields in the temporal cortex 
of the cat. (a) Lateral view, (b) Lateral view that is 'unfolded' to 
show the part of the fields that are normally hidden within the sulci 
(orange shading), as well as the high- and low-frequency limits of 
the tonotopic fields. The four tonotopic fields are the anterior (A), 
primary (AI), posterior (P), and ventroposterior (VP). Positions of 
the lowest and highest center frequencies in these fields are indi¬ 
cated in (b). Note that at the boundaries of the tonotopic fields, the 
direction of tonotopy is reversed so that adjacent fields have 'mirror- 
image' tonotopy. Other cortical fields have less rigidly organized 
tonotopy or little tonotopy. These fields are secondary (All), ven¬ 
tral (V), temporal (T), and dorsoposterior (DP). Also indicated are 
suprasylvian sulcus (sss) and anterior and posterior ectosylvian 
(aes, pes). Source : Brown, 2003. 


techniques for its study, such as transcranial magnetic 
stimulation (TMS) and fMRI. 


3.0 FUNCTIONAL MAPPING OF 
AUDITORY PROCESSING 

Within auditory cortex, are there sub-regions that are 
specialized for decoding different types of sounds 
such as tones versus speech versus music? Or are all 


areas of auditory cortex involved in all sound process¬ 
ing, regardless of the class of stimulus? Are sounds 
processed identically in the left and right hemispheres 
or are there subtle differences? Auditory scientists are 
still unraveling these puzzles. The advent of neuroim¬ 
aging techniques has provided us with new ways to 
investigate brain areas for sound processing. Much 
of this work has been motivated by the investigation 
of brain regions that may be specialized for decoding 
speech and language. While neuropsychological stud¬ 
ies of patients with brain damage have provided a 
wealth of information regarding specific language def¬ 
icits and their correspondence to brain areas, neuroim¬ 
aging allows the investigation of speech and language 
processes in healthy individuals. Neuroimaging also 
provides a way to investigate aspects of auditory func¬ 
tion that have not been able to be addressed before, 
such as what brain areas are involved in imagining a 
sound versus hearing a sound? And what happens in 
auditory cortex while we sleep? 

3.1 Primary auditory cortex 

We have discussed a hierarchical model for sound 
processing, developed in animal studies, with neurons 
in primary auditory cortex tuned to extract basic physi¬ 
cal features in sounds while neurons in non-primary 
auditory cortex are tuned for extracting more complex 
features. Recent studies in humans have provided evi¬ 
dence that this hierarchy is present in human auditory 
function, with basic features in sounds encoded in pri¬ 
mary auditory cortex and more complex information in 
sounds encoded in the planum temporale (Wessinger 
et al., 2001). This area of investigation is still in its early 
stages, and so we must treat these initial findings with 
a bit of caution until we have learned more. Thus, the 
investigation of the functional role of human primary 
auditory cortex is still unfolding, and it is likely that 
human auditory areas serve somewhat differing func¬ 
tions from those found for non-human species. 

3.2 The role of the planum temporale in sound 
decoding 

Early anatomical studies provided evidence for an 
important asymmetry in human auditory cortex: the 
planum temporale (PT) was much larger in the left hemi¬ 
sphere for right-handed individuals (Geschwind and 
Levitsky, 1968). The prevalence of this asymmetry and 
its location in auditory areas close to Wernicke's area 
for speech comprehension motivated the hypothesis 
that PT was the site for auditory speech and language 
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processing. This idea has been supported by neuro¬ 
imaging studies investigating the functional role of PT 
in speech perception. However, neuroimaging studies 
of PT response to different classes of speech and non¬ 
speech sounds provide evidence that the functional 
role of PT is not limited to speech sounds. These find¬ 
ings have resulted in a new assessment of the role of PT 
in sound processing. New light has been shed on the 
role of PT in sound and language processing by Hickok 
and his colleagues. In a recent review of the functional 
neuroanatomy of language, Hickok points out that it 
is perhaps not surprising that the PT seems to respond 
to so many classes of sounds and for such a wide vari¬ 
ety of tasks. If you look at the PT in terms of its cellu¬ 
lar (cytoarchitectonic) structure and organization, then 
you will see that within the anatomical bounds of this 
region lie four distinctly different fields (Figure 7.15). 
These differing fields likely correspond to differing 
functions performed within this area and may explain 
why PT is activated in so many neuroimaging studies 
for a wide variety of tasks and stimulus types. Much 
more investigative work is needed in order to provide 
a thorough account of the role of functional areas in 
human auditory cortex, such as the planum temporale. 



FIGURE 7.15 Just what role does the PT play in sound and 
language perception? Cognitive neuroscientists are still assessing 
the PT's role in auditory processing. A recent review of the func¬ 
tional neuroanatomy of language by Hickok (2009) shows that the 
area we refer to as PT is likely not a unitary functional region at all. 
If you look at the PT in terms of its cellular (cytoarchitectonic) struc¬ 
ture and organization, then you will see that within the anatomical 
bounds of this region lie four distinctly different fields (shown in 
green, red, yellow, and blue). These differing fields likely corre¬ 
spond to differing functions performed within this area and may 
explain why PT is activated in so many neuroimaging studies for a 
wide variety of tasks and stimulus types. Source: Hickok, 2009. 


3*3 Cortical auditory ‘what* and ‘where’ 
systems 

The central role of the auditory perception system is 
to extract information from the listening environment 
in order to determine what is happening around us. 
Consider again the large college classroom where doors 
are opening and closing, students are talking, back¬ 
packs are being unzipped, books are being dropped 
on desktops. All of these sound events are happening 
at the same time and are overlapping in frequency and 
intensity. How does the auditory system decode the 
individual auditory 'objects' such as a friend's voice, a 
door shutting, a cell phone ringing? To accomplish this, 
the auditory system must keep track of many aspects 
of the complex auditory scene: where sounds are occur¬ 
ring in space, when sounds occur - are they simultane¬ 
ous or does one sound precede another? - to determine 
what the sound represents in terms of known auditory 
objects such as speech or music or new auditory objects 
to be learned. Of course, these perceptual tasks are not 
limited to the auditory system but make contact with 
other sensory systems as your brain integrates what 
you hear with what you see, feel, and smell. These 
tasks also interact with your memories and learned 
information already stored regarding auditory objects 
that have taken a lifetime to develop. Let's begin with 
some auditory perceptual processes within the frame¬ 
work of the auditory system. 

3.3.1 ‘ Where 1 system: sound localization 

An important aspect of auditory scene segmentation 
is to know where a sound is coming from. Imagine a 
busy airport baggage area where you are waiting for 
a friend to arrive. The area is full of people talking. 
There are frequent public address announcements: 
this is a complex auditory scene! Eventually your 
friend spies you and calls out: 'I'm over here!' This is 
not extremely helpful to you because you don't know 
where 'over here' actually is, calling out, 'I am 10 feet 
(3 m) behind you and slightly to the right!', would 
seem to be somewhat more useful (although it actu¬ 
ally is not: sound localization computations are much 
faster than speech comprehension). Despite the lim¬ 
ited information in 'over here', you will likely spin 
around and make visual contact quickly. This is your 
auditory system doing what it does best: localizing 
sound even in complex listening environments with 
many competing sounds. 

Sound localization is a fundamental process for the 
auditory system. Knowing where a particular sound is 
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coming from is quite useful in decoding the auditory 
scene, but of course, it is also critical for survival - 
allowing us to jump out of the way of an oncoming car 
or to duck when we hear a loud noise. How does the 
brain locate sounds in space? You may have an intui¬ 
tion that this is a much more difficult process in some 
ways than the way the visual system maps the visual 
scene in space (see Chapter 6). Sounds are always 
changing in time and the mapping of auditory space 
is a complex one. Here is how it works: when a sound 
occurs, it will likely be off to one side or the other of 
you. It could also be behind you. In order to deter¬ 
mine where the sound is in relation to you, your audi¬ 
tory system must make a very quick determination of 
the sound's arrival at the two ears. Two basic types of 
cues are used when our system localizes sound. The 
first is the interaural (between ear) time difference : the dif¬ 
ference in time between a sound reaching your left ear 
versus your right. A second important cue for localiz¬ 
ing sounds is the interaural level difference. This is the 
small difference in loudness that occurs when a sound 
travels toward the head from an angle. The head pro¬ 
duces a 'sound shadow', so that sounds reaching the 
far ear are somewhat quieter than the near ear and the 
absolute level differences depend on the frequency of 
the sound. (See Figure 7.16.) 

Thus, sound localization processes rely on the basic 
notion that if a sound occurs to the left of you, it will 
make contact with the eardrum of the left ear slightly 
before the right ear and it will be slightly louder in the 


(a) Interaural time differences 



left ear than in the right. The actual computations that 
produce sound localization functions involve com¬ 
plex algorithms called head-related transfer functions 
to calculate the location of sounds in auditory space. 
The neural computations that underlie sound locali¬ 
zation are not well understood. One central 'prob¬ 
lem' that these computations must address is the fact 
that the human head changes size rather dramatically 
throughout childhood. The neural code for the head- 
related transfer functions must be able to take into 
account a changing head size. 

Recent investigations show that these computations 
are performed both in subcortical and cortical audi¬ 
tory regions. However, there remains some debate as 
to how these regions interact. For example, in a series 
of MEG studies to evaluate the cortical dynamics of 
spatial processing, Tiitinen and colleagues (Tiitinen 
et ad., 2006) have reported that while both left and right 
auditory areas appear to be involved in decoding spa¬ 
tial information, the right hemisphere response is more 
prominent (Figure 7.17). Does this mean that the right 
hemisphere is specialized for decoding sound localiza¬ 
tion information? Probably not: while there may be a 
subtle bias in the right hemisphere for processing spa¬ 
tial information, more studies are required in order to 
map out how complex and time-varying spatial infor¬ 
mation is decoded in auditory cortex. 

There is, however, general agreement that the inter¬ 
aural time and level differences for sound localization 
are minuscule (fractions of a millisecond) and demand 


(b) Interaural level differences 



FIGURE 7.16 We present a schematic drawing to illustrate the two cues for binaural sound localization. 
The sound source is shown as a solid dot to the right of the cat's head. The sound waves produced by the sound 
are shown as concentric lines. (A) Interaural time differences result because it takes the sound waves longer to 
travel to the ear away from the sound source. (B) Interaural level differences result because the head forms a 'sound 
shadow/ which reduces the level (loudness) of the sound at the ear more distant from the source. Source: Brown 
and Santos-Sacchi in Squire et al., 2008. 
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FIGURE 7.17 The Plm, Nlm, and P2m responses as indicators of the cortical processing of sound source direction. Realistic spatial 
sound was presented from eight equally spaced source directions in the azimuth plane, (a) Grand-averaged MEG responses. Sounds from 
each source direction elicited a response complex comprising the Plm, Nlm, and P2m. The right-hemisphere Plm and Nlm peaked ear¬ 
lier for sound sources contralateral to the hemisphere. The amplitude of the PI m and N1 m in both hemispheres and the right-hemispheric 
P2 m varied according to the sound source direction. Overall, sound sources in the contralateral hemisphere resulted in larger peak ampli¬ 
tudes, and the right-hemispheric responses were larger in amplitude than the left-hemispheric ones, (b) Grand-averaged minimum current 
estimates (MCE) obtained at the latencies of the Plm, Nlm, and P2m to the 3D sound from the direction angle of 90 degrees. Activation 
restricted to the vicinity of the auditory cortex was observed at each latency. Source : Adapted with permission from Tiitinen et al., 2006. 


a system that can decode tiny differences in sounds 
with amazing precision. In fact, the encoding systems 
for localizing sound form the basis of complex hearing 
and likely form the neural underpinnings of speech 
and music perception in humans. 

A ‘where’ system for spatial attention 
While most auditory cortical neurons respond to 
inputs from both ears, the response is asymmetric, 
with a stronger representation of information from 
the contralateral ear compared to the ipsilateral ear. 
Is there a similar effect for attending to different sides 
of auditory space? Early investigations by Hillyard 
and colleagues provided evidence that there is. Using 
event-related potentials (ERPs), Hillyard reported a 
predominant N1 larger response for the attended ear 
in contralateral cortex (Hillyard et ah, 1973). More 
recent neuroimaging studies have also provided a 
way to investigate the effects of selectively attending 
to auditory space, confirming the earlier findings by 
Hillyard with different methodology (Tzourio et al., 
1997; Woldorff et al., 1999). In a PET study, higher lev¬ 
els of activation in right auditory cortex were found 
when subjects attended to the left, and higher levels 


of activation in left auditory were found when sub¬ 
jects attended to the right (Figure 7.18) (Tzourio et ah, 
1997). 

This study is an example of Top down' processing: 
rather than presenting sounds to the left or right side 
of the subject, the investigators merely instructed the 
subject to attend to the left or the right side of audi¬ 
tory space. These results demonstrate the powerful 
role of attention in the auditory system. And, since the 
hemisphere differences are due to the subject's atten¬ 
tion being focused in differing auditory spatial loca¬ 
tions, it also demonstrates the importance of careful 
experimental design when using neuroimaging tech¬ 
niques to investigate auditory cortical processing. For 
example, if you were investigating hemisphere differ¬ 
ences for vocal music versus instrumental music, it 
would be important to make sure your subjects were 
not attending to just the left or just right side of audi¬ 
tory space while listening! 

Expertise in spatial attention networks 
While the role of attention in processing spatial infor¬ 
mation has been studied for many years, we are still 
unraveling some of the mysteries of how our brains 
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FIGURE 7.18 Example of individual PET activation images 
obtained at the Heschl's gyms level (top) and through the superior 
temporal gyrus (bottom) in one subject. Left column, attend to the 
left deviant sounds versus rest; right column, attend to the right 
deviants versus rest. Source : Adapted from Tzourio et al., 1997. 


accomplish the complexities involved in these pro¬ 
cesses. One focus of recent study is investigating indi¬ 
vidual differences in how individuals use attentional 
processes in decoding spatial cues. While this area of 
study is still in its early stages, we highlight a study 
by Munte and colleagues (Munte et al., 2001) where 
musical conductors' use of spatial attention was con¬ 
trasted with other musical individuals (pianists) and 
a control group of non-musicians. The underlying 
notion was that a conductor must be able to attend 
to many spatial locations at once as he or she is lead¬ 
ing a large number of musicians in playing a musical 
score. Munte and colleagues reported that conductors 
showed a higher sensitivity for sounds presented in 
peripheral listening regions than either of the other 
groups (Figure 7.19). 

While these findings are intriguing, more studies 
must take place before we may understand the types 
of individual differences that occur in spatial attention 
processes. 


3.3.2 4 What’ system: auditory object recognition 
and scene analysis 

Our knowledge about sounds and what they mean 
begins before birth (see Chapter 15) and continues 
throughout life as we experience complex auditory 
scenes. You can imagine that the neural processes for 
decoding the college classroom example that we have 
used throughout this chapter did not get established 
overnight, but are the outcomes of years of experience. 
In this section, we will discuss the learning processes 
associated with forming mental representations of 
auditory objects as well as those for decoding compli¬ 
cated listening environments. Let's begin with auditory 
objects. 

Auditory object recognition 

Knowing where a sound is coming from is an impor¬ 
tant aspect of auditory scene analysis and critical for 
survival. The next step is to understand what you are 
hearing. To accomplish this, the auditory system must 
decode sounds 'online' as they occur in order to form 
a percept of a sound event or auditory object. These 
objects are learned over time as we grow from infant, 
to child, to adult, and they change with experience 
throughout our lifetime. Auditory objects can take 
many shapes, similar to visual objects, and vary widely 
in complexity from a simple computer alert chime, 
to the slamming of a car door, to a friend's voice, to a 
symphony. It seems that the brain has a nearly limitless 
capacity for storing and retrieving auditory objects. 
Auditory objects are organized into categories, such 
as human voices, musical instruments, animal sounds, 
that aid us in decoding learned objects as well as learn¬ 
ing new ones. Over time, associations are formed 
between learned auditory objects and coinciding inputs 
from other sensory systems and these different sensory 
memories become integrated in the conceptual repre¬ 
sentation system. Early work on describing how these 
sensory inputs are experienced and combined to form 
conceptual knowledge was provided by Carl Wernicke, 
who proposed that with experience, when you hear 
the sound of a bell, you will recognize it as such and the 
sound of the bell will also bring to mind (activate) 
the visual features of a bell, the feel of a bell, etc. 
(Wernicke, 1874/1977). 

Because language is uniquely human, it is prob¬ 
ably not surprising that there have been many inves¬ 
tigations into how the brain decodes speech. We will 
discuss these later in this chapter. For now, let's focus 
on how the brain decodes non-speech sounds such 
as environmental sounds. Here we highlight a recent 
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FIGURE 7.19 Effects of auditory attention in conductors, pianists, and controls. 

(a) Experimental setup; speakers are spaced 6 degrees apart. Group-average event-related 
potentials (ERPs; frontal midline site) recorded from the conductors and invoked by frequent 
standard stimuli are represented by blue lines that indicate response to stimuli from a partic¬ 
ular speaker when attending to loudspeaker Cl; red lines represent ERPs in response to the 
same stimuli when attending to speaker PI. Attended stimuli give rise to an enhanced negativ¬ 
ity starting at 60 ms. ERPs associated with adjacent speakers show a similar declining gradient. 

(b) Difference waves obtained by subtracting unattended-direction from attended-direction 
responses. All subject groups showed a gradient ERP for central locations, for peripheral 
sounds, a gradient is evident only for the conductors, (c) Top row : Electrophysiological atten¬ 
tion effect (frontal midline electrode, mean amplitude, 180-220ms; Cl/PI set to 100%). No dif¬ 
ferences between groups were found for central locations. Conductors show steeper gradient 
in the periphery. Bottom row : Button presses in response to infrequent stimuli. For peripheral 
sounds, conductors show a decreased false alarm rate for adjacent locations, (d) Spine-interpo¬ 
lated scalp maps of the attention effect for the centermost speaker (time window, 180-220 ms) 
show a similar topography across groups. Source : Adapted from Munte et ah, 2001. 


study by Binder and colleagues (Lewis et al., 2004) who 
investigated brain areas for recognizing environmen¬ 
tal sounds (e.g. the sounds of a doorbell, a hammer 
pounding a nail). Results are presented in Figure 7.20: 
the key finding was that auditory areas in the supe¬ 
rior temporal gyrus were activated by both recognized 
and unrecognized (reversed) environmental sounds. 
However, recognized sounds also activated regions in 
the superior temporal sulcus and the middle temporal 
gyrus (MTG) in both hemispheres. These results are 


interesting in light of previous investigations of the 
functional processes of the MTG: the regions iden¬ 
tified in this study partially overlap with semantic 
systems and with areas that have been reported as 
important for recognition of visual objects such as 
tools. During the process of learning an environmen¬ 
tal sound, it is likely that the sound of a hammer will 
be heard at the same time movement of the hammer 
is seen. Thus, the sound and the sight of hammering 
are likely to be linked during the process of learning. 
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FIGURE 7.20 Brain regions involved in environmental sound recognition. Yellow hues show group-averaged acti¬ 
vated regions and dark blue shows relative decreases evoked by (a) recognizable, forward sounds relative to silence or 
(b) the corresponding unrecognizable, backwards sounds relative to silence, (c) Data from (b) subtracted from (a), revealing 
regions preferentially involved in recognizing sounds (yellow) versus not recognizing the corresponding backward-played 
sounds (light blue), both relative to silence, (d) Flat maps showing data from (c). The left superior temporal sulcus is out¬ 
lined in gray for clarity, (e) Axial sections of data from (c) displayed on the brain of one subject. Source: Lewis et al., 2004. 


Binder and colleagues propose that this is the case and 
that the MTG region is a likely candidate for the brain 
region for this type of object recognition processing. 

The results of the study by Binder and colleagues 
(Lewis et al., 2004) are in good accord with earlier 
studies of individuals with brain lesions who suffered 
from auditory agnosia, the inability to recognize audi¬ 
tory objects. Cases where the patient suffers from a 


specific ('pure') deficit for recognizing environmental 
sounds, while leaving speech recognition intact, are 
rare. The investigations of the cases that do exist have 
reported a complex location of lesions, with reports of 
left hemisphere damage, right hemisphere damage, 
or in some cases bilateral damage (see Clarke et al., 
2002). The results of Binder and colleagues, showing 
auditory object recognition - related activity in several 
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areas in both hemispheres, provides evidence that the 
neural substrates of auditory environmental object 
perception are likely complex and include multiple 
regions in both hemispheres. This work is an example 
of how the results of lesion studies inform scientists 
using neuroimaging techniques to investigate complex 
cortical processes. 

The cocktail party effect 

We have described how a sound is decoded by the 
auditory system to be recognized or learned as an 
auditory object. This process seems relatively straight¬ 
forward when you consider a situation where a single 
sound event occurs in a quiet environment. But how 
is this perceptual task accomplished in noisy environ¬ 
ments, with complex sounds that occur simultaneously 
in time, with overlapping frequencies, and possibly 
coming from the same spatial location? How does the 
auditory system distinguish them as separate sound 
events? This perceptual task - the 'cocktail party prob¬ 
lem' (Cherry, 1953) - has been the subject of many 
investigations of auditory perception from a theoreti¬ 
cal perspective to understand how the auditory system 
extracts information from complex signals as well as a 
practical perspective in designing speech recognition 
systems. Bregman (1990) provided a model to describe 
how the auditory system segregates the many different 
signals in a noisy environment. The four elements in 
this model are: 

1 the source 

2 the stream 

3 grouping 

4 stream segregation. 

The source is the sound signal itself. The stream is the 
percept related to the sound. This distinction between 
the physical signal and the related perception is analo¬ 
gous to the relationship we described earlier in this 
chapter between the frequency (in Hz) of a sound and 
the pitch perceived by the listener: the source represents 
the physical features of the signal which can be well 
described in terms of its frequency, intensity, spatial 
location, etc., while the stream represents the psycholog¬ 
ical aspects which may vary widely across individuals. 

Grouping refers to how the signals are perceptually 
combined to identify and maintain attention to some 
aspects of the auditory scene (such as listening to one 
friend's voice in a crowd of people). Perceptual group¬ 
ing processes create the stream. There are two basic 
types of grouping: simultaneous grouping , where if two 
or more sounds have common onsets and offsets, they 


may be grouped together. Think of a choir or an orches¬ 
tra: you will not typically hear each individual voice or 
instrument, but will group them into a single stream 
due to the beginning and ending of their music together 
as well as their shared spatial location. (On the other 
hand, if you pay particular attention, you can attend 
to a single voice or instrument: this is the dynamic 
auditory system at work!) Sequential grouping refers to 
the process in which features or properties are shared 
across sounds that occur over time. An example of this 
grouping process is if you are listening to a professor 
lecture and someone in front of you coughs. The stream 
coming from the professor is interrupted by the cough 
but you will likely not notice an effect in hearing what 
is being said. This is your auditory system recogniz¬ 
ing that the professor's voice represents a single stream 
despite the brief interruption produced by the cough. 

Stream segregation uses the grouping processes to 
segregate separate auditory objects or events into 
streams. Here are the basic grouping principles that 
Bregman developed in his model: 

1 Proximity: sounds that occur close together in time 
and share common features or properties may be 
grouped together 

2 Closure: sounds that share belongingness will be 
grouped, such as the example of the cough during a 
lecture - the cough does not 'belong' to the stream 
produced by the lecture and is not grouped into 
that stream 

3 Good continuation: sounds that have smooth tran¬ 
sitions are likely to be grouped together (this is 
similar to the proximity principle) 

4 Common fate: sounds that come from the same 
location or coincide in time may be grouped 
together (such as the orchestra example) 

5 Exclusive allocation: this is an important princi¬ 
ple for speech perception and states that one may 
attend to (or allocate neural resources for) one 
stream or another but not both at one time. This 
aspect of auditory processing is frequently referred 
to as selective listening. If one speaker's voice is 
presented to your two ears in headphones, you 
have no difficulty in understanding what is being 
said. If, however, two different speakers' voices 
are presented simultaneously to your left and right 
ears, you will only be able to attend to one stream 
at a time, although you may switch back and forth. 

How does the brain perform auditory scene analy¬ 
sis? Investigations of the neural substrates of percep¬ 
tual organization have led to the formation of several 
theories of how and where perceptual streaming is 
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decoded. One view holds that auditory stream segre¬ 
gation involves primary auditory cortex and that the 
underlying mechanisms for this segregation involve 
neural suppression of information not contained 
within an auditory stream (Fishman et al., 2001). 
A second view holds that auditory stream segmenta¬ 
tion exploits cortical change detector mechanisms in 
detecting aspects of the auditory scene that are not 
part of a single stream (Sussman, 2005). According to 
this view, an individual auditory stream is detected 
based on the acoustic aspects of the auditory sound, 
such as its frequency and location in space. Once these 
characteristics are formed into a neural representation 
of the stream, inputs that do not match this stream 
are detected using auditory cortical change detec¬ 
tion mechanisms. A third view is that the perceptual 
organization processes take place in an area of cortex 
that is thought to underlie binding processes for vis¬ 
ual and somatosensory input, the intraparietal sulcus 
(Cusack, 2005). In this view, the perceptual organiza¬ 
tion of multiple auditory streams occurs external to 
auditory cortex in neural territory that is implicated in 
multimodal cortex (Figure 7.21). 

The segmentation of an auditory scene into its 
constituent parts, or streams, is a complex 'problem' 
for the auditory system to resolve and likely has an 
equally complex neural basis. Thus, it is not surprising 



FIGURE 7.21 Cortical areas for auditory stream analysis: region 
(shown in light green) in the intraparietal sulcus (IPS) when two audi¬ 
tory streams are perceived versus 1. The IPS has been implicated as a 
region for perceptual organization (binding) of multimodal (vision, 
touch, sound) information. Source : Adapted from Cusack, 2005. 


that the neural substrates for auditory scene analysis 
are still being elucidated in the relatively new field of 
cognitive neuroscience. 

3.3.3 ‘What’ and 'where' processing streams 

We have discussed important aspects of auditory per¬ 
ception, including decoding binaural cues to deter¬ 
mine where a sound is in space and extracting spectral 
and temporal features to determine what that sound is. 
'What' and 'where' processing streams have been the 
topic of many invasive investigations in non-human 
primates in both the visual and auditory modalities. 
Converging evidence from these investigations provides 
evidence of distinct and separable streams for process¬ 
ing 'what' and 'where' information (Figure 7.22). 

There is a large and growing body of evidence that 
cortical networks for decoding what and where infor¬ 
mation in sound are processed in separate (but highly 
interactive) processing streams in the human brain. 
The planum temporale has been suggested to serve 
as a hub in early auditory cortical processing making 
contact with two differing streams of processing for 
decoding spatial location and auditory object identifi¬ 
cation (Griffiths and Warren, 2002). However, the role 
of the planum temporale in sound processing is con¬ 
tinuing to be elucidated. 

Recent neuroimaging studies have investigated 
processing of where and what information and have 
shown differing patterns of activity for decoding this 



FIGURE 7.22 Auditory regions and streams in the primate 
brain, (a) The lateral surface of a macaque brain showing regions of 
visual (pink) and auditory (blue) responsivity. Multimodal respon- 
sivity is shown in purple, (b) Two broad "streams' of processing 
within the auditory system. Source: Adapted from Scott, 2005. 
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FIGURE 7.23 Functional responses to speech and candidate 
stream of processing in the human brain, (a) The lateral surface of 
the human brain: the colored regions indicate broadly to which type 
of acoustic signal each temporal region (and associated parietal and 
frontal region) responds. Regions in blue show a specific response 
to language-specific phonological structure. Regions in lilac respond 
to stimuli with the phonetic cues and features of speech, whereas 
those in purple respond to intelligible speech. Regions in pink 
respond to verbal short-term memory and articulatory representa¬ 
tions of speech. Regions in green respond to auditory spatial tasks, 
(b) The putative directions of the 'what', "where', and 'how' streams 
of processing in the human brain. Source : Adapted from Scott, 2005. 

information. In a recent review article, Scott (2005) 
provides a summary of findings to date and pro¬ 
vides hypothesized brain regions for 'what', 'where', 
and 'how' processing streams in the human brain 
(Figure 7.23). However, the functional mapping of cor¬ 
tical auditory processing streams remains an ongoing 
investigation. 

4*0 SPEECH PERCEPTION 

Now let's turn to an important area of investigation in 
the topic of human auditory function: the decoding of 
speech sounds. Since language is a uniquely human 
function, it may well be the case that the auditory 
system in humans differs sharply from those in non¬ 
human primates. Let's begin our discussion of how 
we decode human speech with a brief discussion of 
the units of analysis in speech. 

The basic task of the speech system is to map sounds 
onto meaning. This seems to be a relatively straight¬ 
forward process: when a speech sound, such as'd', is 
heard, the physical sound is mapped onto an abstract 


representation of that sound, the phoneme. The two 
main types of phonemes are consonants (such as'd') 
and vowels (such as 'i'). Individual phonemes are 
stored in echoic memory while an entire word is being 
spoken, for example, 'dig'. In order to decode the spo¬ 
ken word 'dig', you might imagine that the neural rep¬ 
resentations for'd', 'i', and 'g' are decoded individually 
and sequentially, and combined to map onto a sound 
representation of the word 'dig'. The result is that the 
word 'dig' is activated in the semantic/conceptual 
knowledge system. Unfortunately, this description 
makes perfect sense but it is not how the speech system 
actually works. In fact, there is little agreement in the 
field of speech perception regarding the basic 'building 
blocks' of speech: is an individual phoneme the small¬ 
est unit of analysis for speech systems? Or is the sylla¬ 
ble the appropriate unit? We will return to this question 
later in the chapter. For now, consider that the speech 
system must not only decode the individual phonemes 
in speech to map the sound information to meaning, 
but it must also decode 'who' information in order to 
know who is speaking and 'when' in order to under¬ 
stand the temporal order of speech phonemes, syl¬ 
lables, words, and sentences. As mentioned earlier in 
the chapter, the speech signal must be evaluated across 
multiple time scales (20, 200, 2000 ms, see Figure 7.2). 
This information must be decoded accurately regardless 
of the differences in human speech: whether we hear a 
high-pitched voice of a child or a low-pitched voice of 
a man, whether we are speaking very loudly or whis¬ 
pering, or whether we are speaking quickly or slowly. 
Obviously, the speech system is doing a lot more than 
a simple mapping of sound onto meaning and it can¬ 
not rely solely on the physical aspects of speech since 
they vary so widely both within and across speakers. 
Despite the intricacies of speech perceptual processes, 
they occur with little attention or apparent effort on our 
part. Let's begin with a little history of speech science. 

4.1 Background and history 

Research into how speech is decoded is a relatively 
young area of investigation. While scientists have long 
studied language in the brain and investigations of 
the physical properties of sound date back to the early 
19th century, the specific study of how acoustic signals 
map onto meaning is relatively new. Speech percep¬ 
tion investigations began during the pre-World War 
II era. Several events combined to move the study of 
speech perception into the forefront. First, prior to and 
during World War II, there was a need for develop¬ 
ing speech recognition systems for covert war-related 
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BOX 7.1 From vocoder to bionic hearing 


The technology underlying early vocoders remains in active use today and forms the basis for the way in which coch¬ 
lear implants stimulate the auditory system to provide hearing for individuals with certain kinds of hearing losses (see 
Figure 7.24). 
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FIGURE 7.24 Source : Brown, 2003. 


communications. Next, two inventions changed the 
way we thought about and studied human speech. 
First was the vocoder (voice + coder) developed 
by Homer Dudley of Bell Telephone Laboratories 
(Dudley, 1936). The vocoder provided a method to 
transmit speech signals over long telephone circuits 
by analyzing and recoding speech into simpler sig¬ 
nals that contained far less information than natural 
human speech. The outcome was a far simpler speech 
signal that was, nonetheless, quite understandable, 
providing evidence that the speech signal contained 


many redundant features. These findings launched 
experiments to determine what was the minimal infor¬ 
mation required to comprehend speech. 

A second invention was actually developed dur¬ 
ing World War II but kept secret until after the war 
ended: the spectrograph. The spectrograph was also 
developed by scientists at Bell Labs and was based on 
some of the principles Dudley developed during the 
making of the vocoder. The spectrograph analyzed the 
sound signals and produced a picture called a spectro¬ 
gram (Figure 7.25) or visible speech (Potter et al., 1947). 
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FIGURE 7.25 Spectrograms of individual spoken words. Time is depicted on the x-axis and frequency is depicted on the y-axis. Darker 
shadings indicate higher amplitude, mostly contained in the formants within each speech sound. Source : Adapted with permission from Binder 
et al., 2000. 


In a spectrogram, frequency of the speech signal is 
presented on the y-axis and time on the x-axis. The 
shading of the speech signal represents a third dimen¬ 
sion, intensity (or energy). Intensity is presented in 
two ways: across time (x-axis) and within frequency 
(y-axis). In this way, the amount of intensity or energy 
can be represented both as a function of time during a 
sentence and as a function of frequency band within a 
spoken syllable or word. No shading (white areas) in 
regions of the spectrogram along the x-axis indicates 
silence at those points in time. No shading in regions 
up and down the y-axis, within each speech sound, 
indicates no energy at that frequency. Similarly, darker 
shading along the y-axis indicates more energy at that 
frequency band. You will note that each speech sound 
presented in Figure 7.25 has regions of darker shading 
at fairly narrow frequency bands: these are the formants 
that occur in human speech. Formants occur at fairly 
regular intervals and are produced by air that oscillates 
as it ascends through the vocal tract. The formants or 
harmonics differ by individual, based on the size of the 
vocal tract. The spectrograph radically changed how 
speech perception was investigated and provided a 
method for scientists to evaluate substructural elements 
of the speech signal in a way never before possible. 

There were two important outcomes of early inves¬ 
tigations of speech using spectrograms: first, spec¬ 
trograms of sentences showed that gaps or silences 
within sentences did not map onto word boundaries, 
but occurred within words in many cases; and second, 
inspection of the detailed information for individual 
phonemes showed that the formant structure for pho¬ 
nemes, such as /d/, differed sharply depending on 
the following vowel (Figure 7.26). 

These two findings had important ramifications 
on models describing how the brain decodes speech. 
First, gaps or silences in the speech stream do not pro¬ 
vide the speech decoding system with information 
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FIGURE 7.26 Schematic illustration of the direction and 
frequency of formant onsets in the syllables /di/ and /du/, dem¬ 
onstrating that, although the percepts of the two syllables begin¬ 
ning with the sound 'd' will map onto a single phoneme /d/, the 
physical instantiations of the initial /d/ are quite different. Source : 
Carroll, 1999, originally from Liberman, 1970. 

about when a word begins or ends. Clearly, the speech 
system needed to use other cues for word recognition. 
Second, individual phonemes in speech were physi¬ 
cally quite different depending on the phonemes that 
occurred just before and after them. This lack of invari¬ 
ance implied that there were no simple sound tem¬ 
plates in the brain that mapped in a one-for-one basis 
to phoneme identities. 

Despite these complexities regarding how speech 
is decoded, spectrograms enabled speech scientists to 
describe pictorially important features in phonemes, 
such as where in the speech articulation system they 
are produced (place of articulation) and the duration 
of the onset of vocal cord vibration (voice onset time). 
Consonants could now be categorized according to 
these distinctive features (Figure 7.27). 

4*2 Early theories of speech perception 

An important outcome of early speech science was 
the realization that the physical features in individual 
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FIGURE 7.27 Schematic illustrations of the formant patterns 
for distinctive features in classes of speech sounds. Source : Brown, 
2003. 

speech sounds or phonemes did not provide invariant 
information for their decoding. Recall that the work 
of Fourier and Ohm provided a basis for deconstruct¬ 
ing complex sounds into simpler sinusoidal parts. The 
findings of lack of invariance in speech sounds indi¬ 
cated that speech decoding systems must be quite dif¬ 
ferent from those for decoding other types of sounds. 
That is, if speech perception did not entail an analysis 
of the sum of the physical parts, and it clearly could 
not because the physical parts vary widely for a sin¬ 
gle phoneme, then how was it performed? One theory 
was that the neural systems for speech decoding were 
specialized and not part of the general auditory sys¬ 
tem. A strong view of this theory that 'speech was spe¬ 
cial' held that the special systems for speech decoding 
occurred as early as the ear (Liberman et al., 1967). 
The lack of invariance finding led Liberman and col¬ 
leagues to develop the motor theory of speech perception 
(for a review, see Liberman and Mattingly, 1985). This 
theory suggested that speech perception was tightly 
coupled to speech production, specifically the motor 
articulation processes or gestures used in producing 
speech. While the acoustics of phonemes lack invari¬ 
ance, the motor theory held that the articulatory ges¬ 
tures used to produce them were invariant and the 
neural representations of these gestures were accessed 
in speech perception. Other theories for speech per¬ 
ception have been developed since the motor theory, 
and this work is ongoing in speech science, with the 
underlying notion that the speech system must have a 
way to maintain a perceptual constancy across a wide 
variety of physical features in phonemes and words. 


A central debate in speech perception science has 
raged since the work of Liberman and colleagues dur¬ 
ing the 1950s and 1960s: the 'speech is special' view 
holds that speech and language processes are encap¬ 
sulated within a specific language system, i.e. they 
are domain-specific (where the domain is language). An 
alternative view is that speech and language processes 
exploit brain systems in use in general cognition, i.e. 
they are domain-general. For speech perception, early 
work by Tallal and colleagues provided evidence that 
left lateralized language processing was not due to 
domain-specific language organization in that hemi¬ 
sphere but, rather, was due to a domain-general audi¬ 
tory processing bias in the left hemisphere for decoding 
rapidly changing temporal features in sounds, such 
as those contained in speech (for a review, see Tallal, 
2001). These two opposing viewpoints are still the topic 
of ongoing discussion and have not yet been resolved. 
However, it is likely that the neural systems for decod¬ 
ing speech have a combination of domain-specific and 
domain-general processing. 


4.2.1 Units of analysis - the building blocks for the 
speech system 

What are the basic elements or units for decoding 
speech? Is it the phoneme? The syllable? The early 
findings of a lack of invariance in phonemes provide 
evidence against the phoneme as the best unit of anal¬ 
ysis for speech perception, although phonemes are 
clearly critical elements in speech perception. Speech 
contains multiple cues, however, and so it is perhaps 
not surprising that a simple one-to-one mapping of 
phonemes to words does not provide a full descrip¬ 
tion of the processes underlying speech perception 
(see Scott and Wise, 2004, for a review). Recall that the 
auditory system is not a uni-directional system, but 
has complex feedback systems that extend all the way 
to the cochlea as well as parallel interactive systems 
across cortical regions within auditory cortex, across 
the hemispheres, and extending to other sensory and 
memory systems. 

These complex processing pathways clearly aid 
in decoding the speech signal. The syllable as a basic 
unit of speech makes some intuitive sense because of 
the lack of invariance issue as well as the importance 
of decoding syllabic stress when mapping sound 
onto meaning. For example, the words 'melody' and 
'melodic' are very similar in terms of the sequence of the 
phonemes. However, the syllabic stress differences in 
the two words lend an important cue to understanding 
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their meanings. Nevertheless, the field of speech sci¬ 
ence has not agreed upon an answer to the question 
of what is the best unit of analysis for understanding 
speech and this issue must remain unresolved here. 

4.2.2 Minimal information for decoding speech 

Since the speech signal is so complex with many over¬ 
lapping cues, perhaps a better way to understand 
how speech is encoded is to investigate what is the 
minimal information required to comprehend speech? 
This approach was taken by Shannon and colleagues 
(Shannon et ah, 1998) with the central idea that the 
'shape' of the speech signal, i.e. the rising and falling 
fluctuations over time, or temporal envelope, carries the 
minimal information required for decoding speech. 
Shannon presented degraded speech that had no fre¬ 
quency information but differing levels of temporal 
information. Listeners were able to decode continu¬ 
ous speech with remarkable accuracy, indicating that 
while frequency information is obviously important 
in decoding speech, it is not necessary. Shannon's 
work provided evidence that the temporal envelope 
of speech might carry more relevant information for 
its decoding than the fine-grained features found in 
phonemes. 

The lack of general agreement regarding the basic 
building blocks of speech in the field of speech science 
makes the investigation of the neural bases of speech 
perception somewhat more difficult! Nevertheless, 
there have been many studies of how speech and 
other classes of sounds are decoded in the brain and 
we highlight some of these here for you in the follow¬ 
ing section. 


4.3 Functional mapping of speech-specific 
processes 

Early neuroimaging studies by Binder and colleagues 
(for a review, see Binder, 1997) investigated stimulus- 
based differences in auditory cortex by comparing brain 
activation in response to speech sounds (words) versus 
tones or noise. A general finding was more widespread 
activation in superior temporal gyrus and the superior 
temporal sulcus for words as compared to the tones or 
noise. Although these results could be interpreted as 
representing speech-specific processing in those audi¬ 
tory regions, they were difficult to interpret, however, 
because words and the non-speech sounds (tones, 
noise) differed not only in terms of representing speech 
versus non-speech classes of sounds, but also in their 
complexity. Therefore, different brain activation pat¬ 
terns might reflect speech versus non-speech functional 
areas, but might also reflect areas that differ in terms of 
decoding complex features in sounds. 

A recent investigation addressed these issues with 
the presentation of many classes of sounds, including 
noise bursts, tones, words, pseudowords (pronounce¬ 
able non-words, such as 'hig', and reversed speech 
(Binder et ah, 2000). The aims of this study were to 
investigate auditory cortical regions that were activated 
for speech versus non-speech, and compare regions 
that were activated for words versus pseudowords ver¬ 
sus reversed speech (Figure 7.28). 

The major findings were that Heschl's gryus and 
the planum temporale were activated similarly for 
all sound stimuli. This result supports the notion that 
sound is processed in a hierarchical fashion, with 
Heschl's gyrus showing activation for all classes of 


Left Right 



FIGURE 7.28 Comparison between three Speech-Tones contrasts (Word-Tones (W-T), Pseudowords- 
Tones (P-T), and Reversed-Tones (R-T)). Areas responding more to speech than to Tones are very similar 
for all contrasts. Source : Adapted from Binder et ah, 2000. 
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sounds and likely representing an early sensory anal¬ 
ysis. Speech sounds activated a larger region of audi¬ 
tory cortex than the non-speech sounds, extending into 
the posterior superior temporal gyrus and the superior 
temporal sulcus. Interestingly, the activation did not 
differ for words, pseudowords, and reversed speech. 
These findings indicate that speech activates a larger- 
scale network of auditory cortex than the simpler noise 
bursts and tones. Because there were no differences 
between the words, pseudowords, and reserved speech 
conditions. Binder and colleagues concluded that these 
regions likely do not reflect semantic processing of the 
word meaning, but reflect phonological processing of 
the speech sounds. 

There are many active lines of investigation into 
the neural mechanisms and regions employed in 
decoding human speech and this work is ongoing. 
Let's move ahead to a discussion of the relationship 
between 'hearing' and 'saying': speech perception ver¬ 
sus production. 

4.4 The link between speech perception and 
production 

Early neuroimaging studies investigated brain activa¬ 
tion for hearing versus producing speech. One impor¬ 
tant finding that has been reproduced many times 
is that auditory cortex is activated during speech 
production tasks as well as during the perception of 
speech (Figure 7.29). 

Why is auditory cortex active during speech pro¬ 
duction? Is it simply the case that while producing 
speech, we hear our own voice? Or does the audi¬ 
tory system play a role in speech production? There 
is evidence from neuroimaging studies, as well as 
from lesion studies with patients with aphasia, that 
speech perception and production systems are tightly 



Red; Speech production (Name Si Say OK > Object & circle decision ) 

Yellow: Speech perception (Listen speech > reversed speech) 


FIGURE 7.29 Red shaded areas show activation for speech 
production while yellow shaded areas show activation for listening 
to speech. Source : Adapted from Frackowiak, 2004. 


coupled. From infancy, as speech and language are 
acquired, there are complex interactions between 
heard language and spoken language which guide 
the development of language. In fact, Carl Wernicke 
proposed a model for language processing in the late 
19th century that included a pathway from auditory 
speech perception areas to motor speech production 
areas and proposed that the 'sound images' of words 
would serve to constrain the output when producing 
words. This model remains in use today, and while we 
know more about dynamic brain processes now than 
during Wernicke's time, the model has provided an 
important theoretical framework for studying human 
language systems. 

4.4.1 Inner speech 

What about brain areas for speech production when 
actual speech is not actually produced, when we talk 
to ourselves? The investigation of brain areas involv¬ 
ing inner speech provides an intriguing way in which 
to study our own consciousness. In a recent article, 
Hesslow (2002) addressed this issue and proposed a 
simplified model of brain areas that may be employed 
for inner speech (Figure 7.30). 

While there are clearly complex interactions between 
brain areas for decoding speech and producing speech, 
theorized in early motor theories of speech perception 
and realized in later brain studies, the exact nature of 
the integrative processes and neural territory that are 
shared during listening versus producing speech are 
still being elucidated in the field of human language 
research. 


4.5 Damage to speech perceptual systems 

Prior to the advent of neuroimaging, much of what we 
learned about brain systems for decoding speech came 
from investigations with individuals who had brain 
damage, typically due to a stroke since they produce 
damage that is more limited in brain area than, for 
example, a closed-head injury. Strokes, nevertheless, can 
vary widely in the amount of brain area affected. The 
result of the stroke is a blockage of blood flow, which 
causes neuronal death and produces a lesion. The 
lesion in the area affected by the stroke, in turn, pro¬ 
duces behavioral symptoms due to the brain damage. 
When a stroke patient has impaired language function, 
it is called aphasia. Patients with aphasia have widely 
varying symptoms depending on the location and the 
size of their lesion. Two basic classifications of aphasia 
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FIGURE 7.30 Internal simulation of conversation, (a) We can 
respond to a question without being conscious of our behavior. 
The verbal signal enters the primary auditory cortex (A) and then 
Wernicke's area (WA). This will elicit formation of a reply in Broca's 
area (B) and the primary motor cortex (M). (b) We can also listen 
and respond to our own talk using the same brain regions, (c) If the 
preparation of the verbal response can be fed directly (red arrow) 
into auditory cortex or Wernicke's area, we can also speak silently to 
ourselves using essentially the same mechanisms. Source : Adapted 
from Hesslow, 2002. 


come from 19th century neuroanatomical investiga¬ 
tions of brain areas for language: Paul Broca discovered 
a region in the inferior frontal lobe that was important 
for speech production, and Carl Wernicke discovered 
a region in the temporal lobe that was important for 
speech perception. Damage to these regions would pro¬ 
duce aphasia with symptoms that differed depending 
on which area was affected: an individual whose major 


19th Century Neurological 


Broca: Speech production 

Wernicke: Speech perception 



Arcuate 



FIGURE 7.31 Upper panel shows classical language areas 
adapted from 19th century neuroanatomists. Lower panel shows 
contemporary view of the classical models. Source : Adapted from 
Frackowiak, 2004. 


symptoms are impaired speech production is classified 
as a Broca's aphasic, an individual whose major symp¬ 
toms are impaired speech comprehension is classified 
as a Wernicke's aphasic (Figure 7.31). 

Blumstein and colleagues (for a review, see 
Blumstein, 1997) used careful testing to determine 
which aspects of speech comprehension were affected 
in aphasia. One series of experiments was designed 
to determine if speech comprehension problems were 
due to damage to speech perceptual processes for 
mapping the acoustic sound onto abstract phoneme 
representations (phonemic deficit ) or if they were due 
to word recognition processes for mapping the pho¬ 
nemes onto meaning (semantic deficit). Blumstein used 
an auditory word-to-picture matching task to test 
the aphasic patients. In this task, the patient hears a 
word and must point to the picture that matches the 
word. The picture choices include a picture of the 
word that was heard (the target) and three incorrect 
choices (foils): one foil is similar in its sound (phone¬ 
mic), one is similar in meaning (semantic), and one is 
unrelated to the target word in either sound or mean¬ 
ing. In Figure 7.33, we recreate this experiment: the 
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BOX 7.2 A new look at a classical model: tractography reconstructions of language pathways 


Recent developments in neuroimaging have provided 
us with new ways to visualize the brain. One major neu¬ 
ral 'highway' of the brain that has long been of interest 
to language researchers is the arcuate fasciculus: this large 
fiber tract connects the anatomical regions that corre¬ 
spond to the functional Broca's and Wernicke's areas 
in the left hemisphere. Disruption of this pathway was 
thought to be the underlying cause for a type of acquired 
language impairment (aphasia) called conduction aphasia. 
However, when carefully tested, individuals with a diag¬ 
nosis of conduction aphasia show a wide array of symp¬ 
toms, raising the question of whether a single type of 
neural damage produced them. 

Researchers Catani and colleagues used diffusion ten¬ 
sor tractography to reconstruct the three-dimensional 
pathways of white matter tracts that course between 
Broca's and Wernicke's area (Catani et ah, 2005). They 


found evidence for the white matter tract known as the 
arcuate fasciculus that formed a direct pathway between 
Broca's and Wernicke's areas, shown in red in Figure 
7.32. However, they also found an indirect pathway that 
runs side-to-side (laterally) with an anterior segment 
(shown in green in Figure 7.32) that connects Broca's 
area with a region in the inferior parietal cortex that is 
known as Geschwind's territory and a posterior segment 
(shown in yellow) that connects Geschwind's territory 
and Wernicke's area. 

Although this new finding awaits replication, it may 
well be the reason that conduction aphasia patients show 
such differing ranges of symptoms: if parallel direct and 
indirect pathways connect Broca's and Wernicke's areas, 
then language disruption would differ sharply depend¬ 
ing on which pathway and where in that pathway the 
damage has occurred. 



FIGURE 7.32 Right panel shows the fiber tracts that connect the functional language areas of Wernicke's area in the temporal lobe 
and Broca's area in the frontal lobe in the left hemisphere. These tracts are reconstructed using tractography and MRI images. A direct 
pathway between Broca's and Wernicke's areas, shown in red, corresponds to the classical description of the arcuate fasciculus that has 
been long thought to be the major connective 'highway' between these important language areas. They also found an indirect pathway: 
the anterior segment connects Broca's area with a region in the inferior parietal lobe known as Geschwind's territory (shown in green) 
and a posterior segment that connects Geschwind's territory and Wernicke's area (shown in yellow). Left panel shows where these tracts 
are relative to the anatomical structures of the left hemisphere. Source : Adapted, with permission, from Catani et al., 2005. 


left panel shows examples of target words 'keys' and 
'peas' which differ in the place of articulation of the 
initial consonant but otherwise share distinctive fea¬ 
tures (voicing: they are both voiceless, manner: they 
are both stop consonants). The right panel shows rep¬ 
resentational picture choices, with a semantic foil (car¬ 
rots) on the top left, an unrelated picture on the top 


right, a phonemic foil (keys) on the bottom right, and 
the correct choice, peas, on the bottom left. 

The pattern of errors (pointing to the incorrect pic¬ 
ture) made by the patient provides information about 
which decoding processes are damaged. If the patient 
makes mostly phonemic or sound-based errors, 
then the inference is that processes underlying the 
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FIGURE 7.33 Left panel illustrates the minimal pairs of words that have initial consonants that differ in a single distinctive feature. In 
this case, the initial /k/ and /p/ differ in place of articulation (velar versus bilabial) but are matched for voicing (voiceless) and manner (stop 
consonant). Right panel depicts the stimuli used for a single trial in an auditory-word-to-picture-naming task. The subject hears a word, in this 
case "peas', and is asked to point to the picture that matches the word. Upper left panel shows a meaning-based (semantic) foil, upper right 
panel shows an unrelated foil, lower left panel shows the correct (target) response, and lower right panel shows a sound-based (phonemic) foil. 


mapping of acoustic sound onto phonemes are 
impaired. On the other hand, if the patient makes 
mostly semantic errors, the inference is that sound 
mapping processes are intact and the speech recogni¬ 
tion problems are due to impairments in systems for 
mapping phonemes onto meaning. Two important 
results of these investigations are that the pattern of 
errors made by patients did not correspond in a mean¬ 
ingful way to their aphasia classification: for example, 
one might suppose that Wernicke's aphasics would 
make many more phonemic errors, due to damage 
in speech comprehension brain areas, than Broca's 
aphasics. This was not the case, however. The second 
important result was that all aphasics - Broca's and 
Wernicke's - did quite well on this task. The conclu¬ 
sion drawn was that speech perception, the ability to 
map the acoustic features of a sound onto representa¬ 
tional phonemes, is largely intact in aphasia. The defi¬ 
cits in comprehending speech, therefore, are not due 
to sound-based impairments. These and other investi¬ 
gations of brain areas that subserve speech perception 
provide evidence that, while language function tends 
to be lateralized to the left hemisphere in most right- 
handed individuals, speech perceptual processes may 
be organized bilaterally, i.e. in both hemispheres. 

Subsequent studies have provided further evidence 
to support this conclusion: in order for an individual 
to show severe speech perception impairment (called 
'pure word deafness'), brain damage must include 
lesions in both hemispheres or lesions in the left hemi¬ 
sphere that include damage to the corpus callosum. 


prohibiting right hemisphere information flow to the 
left (see Poeppel, 2001, for a review). 


4*6 A working model for speech perception in 
the brain 

For most right-handed individuals, language is lat¬ 
eralized to the left hemisphere. The language system 
is not unitary, however, and includes many compu¬ 
tational stages from decoding the speech sounds to 
form an abstract representation, to making contact 
with semantic and grammatical systems, to producing 
speech. A recent model of the auditory language sys¬ 
tem proposes that early speech decoding processes are 
organized bilaterally in left and right auditory fields 
while later semantic/syntactic processes are orga¬ 
nized in the left hemisphere (Figure 7.34) (Hickok and 
Poeppel, 2007). 

While we have learned a lot about how speech is 
processed in the brain, through neuroimaging stud¬ 
ies with healthy individuals and neuropsychological 
studies of individuals with brain damage, our under¬ 
standing of how speech is perceived is still an ongo¬ 
ing investigation. Perhaps this is not surprising since 
speech perception scientists have not agreed upon 
the basic units of analysis for speech perception! New 
techniques, such as TMS, and innovative experimen¬ 
tal designs are providing new data for understanding 
how we decode speech and where in the brain these 
systems are located. 
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Via higher-order frontal networks 




FIGURE 7.34 (a) Shows a schematic for the model of auditory language processing proposed by Hickok 

and Poeppel, 2007. (b) Shows brain regions proposed to reflect stages of the model. Note that early speech 
perceptual systems (shown in green and yellow) for mapping the acoustic-phonetic information in sounds 
onto meaning are proposed to be mediated bilaterally in left and right hemispheres while later processes are 
proposed to be mediated by left hemisphere regions. Source : Adapted from Hickok & Poeppel, 2007. 


FRONTIERS OF COGNITIVE NEUROSCIENCE 


Music and the brain 



FIGURE 7.35 Aniruddh D. Patel, PhD, Neurosciences 
Institute, San Diego, CA, USA. Source : Tom Cogill. 


Music, like language, occurs in every human culture and 
extends deep into our species' past. The oldest known 
musical instruments, flutes made from bird bones and 
mammoth tusks, are over 35 000 years old (Conard et al., 
in press). Although the power of music in human life 
has fascinated thinkers since Plato, only recently has 
research on music and the brain really begun to flourish 
(e.g. Peretz & Zatorre, 2003). 

Several basic facts are already apparent. First, the 
study of brain-damaged patients with altered music per¬ 
ception has shown that music is a highly complex cogni¬ 
tive system with many distinct subcomponents (Stewart 
et al., 2006). Second, neuroimaging research with normal 
individuals has shown that listening to or making music 
engages much of the brain (not just the auditory areas), 
even in people with no formal musical training (Koelsch 
et al., 2005). Third, learning to play an instrument affects 
brain structure and function at both the subcortical 
and cortical levels, and improves nonmusical cognitive 
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FIGURE 7.36 Snowball, a medium sulfur-crested cocka¬ 
too ( Cacatua galerita eleonom ) that synchronizes to the beat of 
human music (Patel et al., 2009). Source: Irena Schulz. 


functions (Schellenberg, 2005; Wong et al. 2007; Hyde 
et al., 2009). Fourth, music processing represents an 
intimate relationship between cognition and emotion 
(Huron, 2006; Steinbeis et al., 2006). 

As with any other branch of science, research on 
music and the brain often is driven by debates. One 
debate in which I have been involved concerns the issue 
of "modularity"; that is, the extent to which music uses 
dedicated brain circuitry vs. shared processing mecha¬ 
nisms with other cognitive domains, such as language 
(Peretz, 2006). Music clearly involves a certain degree 
of brain specialization, but my work and that of several 
colleagues suggest that music and language processing 
overlap to a surprising degree (Patel, 2008). This has 
implications for a number of issues in cognitive neuro¬ 
science, including the study and treatment of language 
disorders (Patel et al., 2008; Schlaug et al., 2008). 

Another active debate concerns evolution. Did 
human music evolve as a biological adaptation because 
it helped our ancestors survive? Or is music a human 
invention, based on brain mechanisms that evolved to 
serve other functions? These questions can easily lead to 
armchair speculation, but one recent trend has been to 
take these questions into the lab by doing cross-species 
research on music perception (Hauser & McDermott, 
2003). For example, my lab recently demonstrated that 
a nonhuman animal can move to the beat of human 


music (Patel et al., 2009; see Figure 7.36). This suggests 
at least one fundamental aspect of music cognition does 
not require brain circuits shaped by natural selection for 
music. 

A major question for future research is also one of 
the oldest questions about music: why does it have such 
emotional power for humans? The biological basis of 
our emotional responses to music are just beginning to 
be explored (Juslin & Vastfjall, 2008), and exciting dis¬ 
coveries will be made in the years ahead. 
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5*0 MUSIC PERCEPTION 

Like speech perception, music perception is uniquely 
human. There are many similarities in speech and 
music perception: music has complex phrase struc¬ 
tures and its perception involves the mapping of 
sound onto meaning (and emotion). Music perception 
allows for the recognition of melodies despite differ¬ 
ences in instruments, keys, and tempos; thus it cannot 
be a system built on absolutes but must have relative 
representations. Thus, music perception systems must 
have the ability to maintain a perceptual constancy in 
music representation. A central difference between 
speech and music perception is that all typically 
developing humans master speech perception. We are 
not only good at speech perception, we are masters! 
This is not the case in music perception: there is tre¬ 
mendously more variability in music perception abili¬ 
ties and significantly more explicit learning that goes 
along with musical acuity. The variability in music 
perception abilities combined with the many levels 
of musical training and skill has made the study of 
music perception difficult because of these inherent 
individual differences. These difficulties, however, 
provide a unique opportunity in that they provide an 
opportunity to understand the effects of learning and 
plasticity in the brain areas that decode music. 

5 *1 Stages of music processing 

The perception of features in music involves many 
stages of processing within the auditory system as well 
as across brain regions (Figure 7.37) (Zatorre, Chen, and 
Penhune, 2007). This processing must include feedback 
as well as feedforward systems as well as making con¬ 
tact with stored memories and experiences as well as 
emotional systems. Music perception is quite different 
from speech perception in that many musical signals 
do not contain any lyrics. Thus, the music perception 
processes likely have a more abstract (non-linguistic) 
representational basis. 

While the music signal is complex, like all sound, 
music has basic physical elements: frequency, intensity, 
and time. The psychological aspects of frequency and 
time in music correspond to pitch (melody) and tem¬ 
poral structure (rhythm). Traditionally, melodic and 
temporal aspects of music have been investigated as 
separate features of music perception. However, they 
likely are not completely independent. Just as some 
speech scientists propose that speech may be pro¬ 
cessed in brain areas specialized just for speech, music 


scientists have theorized that there may be neural sys¬ 
tems specialized for music. Evidence in support of 
music-specific systems in the brain has been provided 
in neuropsychological studies with patients who have 
suffered brain damage. Peretz and colleagues have 
provided a series of investigations with brain dam¬ 
aged individuals showing that, in some individuals, 
pitch or melody perception may be selectively dam¬ 
aged, leaving temporal structure perception intact, 
while in other individuals temporal perception may 
be damaged while pitch perception is intact. These 
findings have led to the development of a model for 
the brain organization for music perception (Peretz 
and Zatorre, 2005), where melodic features in music 
are processed preferentially in the right hemisphere 
and can be selectively impaired with right hemisphere 
brain damage, whereas temporal structure in music 
is decoded in a larger network of brain areas in both 
hemispheres. 


5*2 A separate system for music perception? 

Is music perception a separable aspect of auditory 
processing? While the work of Peretz and colleagues 
provides compelling evidence that this is the case, a 
recent review (Koelsch, 2005) of neuroimaging stud¬ 
ies of music perception describes a growing body of 
evidence in support of the view that some aspects of 
music perception, notably the musical structure or 
syntax and the musical meaning or semantics, share 
neural territory with brain areas for language process¬ 
ing (Figure 7.38). 

While there is ample evidence from the work 
of Peretz and colleagues (see Peretz and Zatorre, 
2005, for a review) that music perception may be 
selectively - and differentially - damaged with lesions 
in left or right hemispheres, the studies reviewed by 
Koelsch (2005) provide compelling evidence for at 
least some shared systems in music and language 
processing. Language and music are both uniquely 
human and highly structured signals, with multiple 
dimensions along spectral and temporal axes for 
understanding their basic and complex structures. 
Perhaps there is no unitary brain region for either 
language or music: it may be the case that language 
and music systems have some neural territory that is 
specific for their processing and some neural territory 
that is shared. Complicating an already complicated 
issue are the differing amounts of musical training 
and levels of expertise among humans, musicians, and 
non-musicians. 
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FIGURE 7.37 Top panel : Have you ever wondered what areas of your brain Tight up' when you play or hear music? Of course, the auditory 
regions do activate when you play an instrument, for example, but so do many motor regions involved in the production of the sound from the 
instrument. In fact, these motor and sensory (auditory) systems are tightly coupled and together form a neural circuitry that provides feedback 
and feedforward information for musical instrument playing. Bottom panel : Results from several neuroimaging studies provide evidence for a tight 
coupling between activity in auditory and premotor cortex: (a) People without musical training were taught to play a simple melody on a keyboard 
and their brain activity for listening to that melody was compared in pretraining vs. posttraining scans. Results showed activity in auditory cortex, 
as expected, when listening to the melody, but there was also activity in the premotor cortex but only in the posttraining condition, (b) In other 
studies, researchers compared the brain activity of musicians while they listened to a piece they knew how to play (left column) with the brain 
activity while they played the same piece but without auditory feedback (middle column). There was significant overlap in the premotor cortex and 
in auditory cortex, suggesting that auditory and motor systems interact closely during both perception and production. Source : Zatorre et al., 2007. 
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x = ±43.35, y = -34.25, z = -3.3 
q (left/right) = 40.8/30.5 nAm 

Music 


x = ± 44.75, y = —36.95, z = -2.65 
q (left/right) = 57.3/49.1 nAm 

FIGURE 7.38 Top panel shows brain areas active for lan¬ 
guage. Bottom panel shows brain areas active for music. Source : 
Adapted from Koelsch, 2005. 


6.0 LEARNING AND PLASTICITY 

A central theme in the study of human cognition 
has been the investigation of how new information 
is encoded in the brain during learning and how the 
brain adapts and reorganizes to new situations or, 
following damage, plasticity. These issues are of theo¬ 
retical value in understanding brain function but also 
have important practical relevance. One key question 
is how the auditory cortex responds to deprivation 
of sensory input due to acquired hearing loss, neural 
damage, or deafness. For example, if a child is born 
profoundly deaf and is fitted with a cochlear implant 
at age 2, will his auditory cortex be receptive to sound 
and will he hear normally? Or will the 2 years of 
no exposure to sound limit the effectiveness of the 
implant? These and other questions regarding cortical 
plasticity are under intense investigation by auditory 
scientists. 


6.1 Plasticity due to deprivation 

Much of what we have learned about the plasticity of 
the auditory system due to deprivation comes from 
animal studies. Recall that the cochlea and brainstem 
are organized tonotopically and that this organization 
is also present in auditory cortex. In animal studies of 


neural plasticity after deprivation, specific areas within 
the cochlea or in the brainstem are lesioned so that a 
range of frequencies will no longer be encoded and 
transmitted to auditory cortex. Following lesioning, 
the organization of auditory cortex is studied to deter¬ 
mine if there are frequency specific changes, reflecting 
neural plasticity Irvine and colleagues have conducted 
many studies using this general approach and reported 
evidence that the cortical frequency maps do indeed 
undergo changes following the lesioning (Rajan et ah, 
1993). We cannot ethically lesion human cochlea or 
brainstems and so our studies of plasticity following 
deprivation in humans must be accomplished in a non- 
invasive manner. While these studies are still in the early 
stages, there is evidence that adults with sudden onset 
high frequency hearing loss have some changes in neu¬ 
ral population response in auditory cortex, implying that 
cortical organizational changes occur following hearing 
loss in humans in a manner similar to findings in animal 
studies (Dietrich et ah, 2001). However, these adults had 
sudden onset hearing loss and far less is known about 
slow onset hearing loss that may occur over many years 
or about more subtle forms of hearing loss. 

What about children who are born with partial or 
complete deafness? When they are fitted with hearing 
aids or implanted with cochlear implants, will they 
develop normal hearing? Eggermont and colleagues 
have investigated the responses in auditory cortex in 
typically developing children and children with hearing 
loss who had cochlear implants (Ponton et ah, 1996a, b). 
The implanted children showed some maturational lag 
compared to the controls. However, following implan¬ 
tation their auditory system continued to mature in a 
typical fashion, showing evidence for plasticity in the 
implanted children. These results are heartening in that 
they show that auditory cortex may develop in a typi¬ 
cal way even if there is deprivation early in life. 


6.2 Plasticity due to learning 

Our auditory system is constantly exposed to novel 
sensory inputs that must be decoded in order for us to 
interpret our listening environment. New voices, musi¬ 
cal melodies, and environmental sounds are learned 
every day. What are the brain mechanisms for learning 
new sounds? How are these new sensory memories 
formed and where are they located in the brain? These 
questions have been the topic of years of investigation 
by scientists using animal models for understanding 
learning and plasticity in the auditory system. While 
some scientists hold that sensory-based memories are 
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FIGURE 7.39 Examples of the effects of training on 
representational area. Organization of center frequencies 
(CFs) in A1 (primary auditory cortex) for a representa¬ 
tive untrained naive (a) and a trained experimental (b) rat. 
Each V indicates an electrode penetration, with colored 
polygons indicating the estimated AI are representing the 
CF according to the color bar shown below the maps; D, 
dorsal. A, anterior. Cortical area representing CFs within 
the conditioned stimulus (CS) octave band, i.e. 4.1-8 kHz, 
is highlighted by outlined polygons and vertical hatching, 
(c) Learning curves for three animals showing low (blue 
squares), mid (black diamonds), and high (red triangles) 
levels of motivation level, (d) Corresponding distributions 
of relative representational area (percent of total Al area) 
for each animal, together with mean naive areas (gray line), 
are shown. Vertical bars on naive area distribution indicate 
+ / — SEM; dashed circle and light gray box highlight the CS 
bin. Source : Adapted from Rutkowski and Weinberger, 2005. 


formed and stored in central memory systems, oth¬ 
ers suggest that these sensory-specific memories are 
formed and stored within the sensory area in which 
they were learned. For example, Weinberger and col¬ 
leagues (see Rutkowski and Weinberger, 2005, for a 
recent review) have developed a model for auditory 
learning that holds that the changes in neural tuning 
for new and relevant sounds happen almost imme¬ 
diately, within a few trials of training, and occur in 
primary auditory (Al) cortex. According to this view, 
the neural tuning of neurons in Al changes to reflect 
the features in sounds that are behavior ally relevant. 
Using classical conditioning experimental paradigms, 
Weinberger and colleagues presented tones paired 
with mild electrical shock. After conditioning, the rep¬ 
resentational maps of Al reflected a reorganization, 
with more neural area devoted to encoding the fre¬ 
quency of the paired tone (Figure 7.39). 

6.3 Plasticity due to expertise 

Non-invasive behavioral studies show similar patterns 
for learning in humans in that learning changes occur 
fairly rapidly and are relatively long lasting. One aspect 
of learning that has intrigued scientists is whether 
highly trained musicians have a different kind of brain 
than unskilled individuals. Certainly, the musicians 
have spent more time and effort on musical training - 
does this change the way their auditory cortex is tuned? 


The work of Rupp and colleagues provides evidence 
that it does (Schneider et al., 2002) (Figure 7.40). 

The notion of brain changes due to situation-specific 
experiences such as hearing loss or deprivation, learn¬ 
ing, and expertise is an intriguing one and there are 
many studies ongoing to investigate the correspondence 
between experience and mechanism, mind and brain. 


7.0 AUDITORY AWARENESS AND 
IMAGERY 

We can close our eyes and shut out visual images but 
we cannot close our ears to shut out auditory events. 
What is the effect on the auditory system? The audi¬ 
tory system is the last sensory system to fall asleep 
(or become unconscious with sedation) and the first 
to awaken. In this section, we highlight some recent 
studies of auditory awareness during less-than- 
conscious states, such as sleep or sedation. We also 
highlight studies of auditory activation for imagined- 
not-heard sounds. These aspects of auditory process¬ 
ing are in their infancy, however, brought about by the 
advent of neuroimaging techniques to measure brain 
responses that are not easily accessed using other tech¬ 
niques, and so much more work must be done before 
we can state definitively how auditory awareness and 
imagery are instantiated in the brain. 
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FIGURE 7.40 The neurophysiological and ana¬ 
tomical data show a large increase in professional musi¬ 
cians and a smaller increase in amateur musicians. 
Left : Dipole strength of the primary cortical response 
at 500 Hz. Sources in the right (thick lines) and left 
(thin lines) hemispheres are superimposed. Right: 
Highlighted areas show the Heschl's gyrus for each sub¬ 
ject, aligned in the same order as the primary evoked 
responses. Source : Adapted from Schneider et at., 2002. 


7*1 Auditory awareness during sleep and 
sedation 

Think about the best way to wake up a sleepy friend - 
call his name! A neuroimaging study investigated brain 
responses in sleep and in wakefulness in two condi¬ 
tions: neutral, where the sound was a simple beep, 
and significant: where the sound was the subject's 
own name (Portas et al., 2000). Two main results of that 
study were that beeps and names activated auditory 
cortex both when the subject was awake and when 
the subject was sleeping, indicating that auditory cor¬ 
tex processes sounds even during sleep (Figure 7.41). 
A second key finding was that auditory cortex response 
for the neutral tone versus the subject's name did not 
differ during sleep, indicating that auditory processing 
during sleep encodes the presence of sounds but did 
not differentiate between these very different sounds. 
Brain activation patterns for names versus tones did 
differ, however, in middle temporal gyrus and frontal 
lobe regions. There were also areas in the amgydala 
that were more active during the presentation of the 
subject's own name during sleep than when awake. 
Do these brain areas represent a circuit in the brain that 
alerts us to wake us up when we hear our own name? 
More investigations are needed to support this theory, 
but, these findings provide intriguing evidence for 
how the auditory system 'wakes itself up'. 

A related finding was recently reported by Zatorre and 
colleagues (Plourde et al., 2006) in a study investigating 


the effects of anesthesia on surgical patients. Many recent 
studies have investigated the level of auditory awareness 
in surgical patients who are anesthetized, with the pri¬ 
mary motivation of ensuring that anesthetized patients 
do not hear sounds in the operating room that may be 
upsetting to them. Results of the study by Zatorre and 
colleagues are similar to those reported by Portas and 
colleagues: auditory cortex was activated by simple 
and complex sounds during anesthesia and response 
patterns did not differ for speech versus non-speech 
sounds (Figure 7.42). 

The authors interpreted these findings to indicate that 
while auditory cortex responds to the presence of sounds, 
even during sedation, the response is non-specific. 
This is a similar finding to the results for beeps and 
names during sleep and wakefulness. However, in the 
anesthesia study, the authors suggested that their find¬ 
ings of no differences across types of sounds meant that 
complex (semantic, emotional) processes engaged dur¬ 
ing conscious states were not activated during sedation. 
Which study is right? More investigations with similar 
stimuli and experimental designs must be conducted in 
order to determine if there are brain areas that are active 
during sleep or anesthesia and that might reflect the 
brain's monitoring of auditory events. 

7*2 Auditory imagery 

'Sounds not heard' are playing in our heads all day. 
Some sounds are uncalled for; they just seem to 
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Beep vs. rest during WAKEFULNESS Beep vs. rest during SLEEP 



FIGURE 7.41 Upper panel: Brain areas active for beep versus rest during recorded dur¬ 
ing wakefulness and sleep. Lower panel : Brain areas for name versus rest recorded during 
wakefulness and sleep. Source : Adapted from Portas et al., 2000. 



FIGURE 7.42 Group average responses for all sounds-silence. 
Activation maps overlaid over average anatomical images. The 
right side of the images corresponds to the right hemisphere. Line 
diagrams show the mean signal amplitude (difference in effect 
size between the two conditions, i.e. sound versus silence). Source : 
Adapted from Plourde et al, 2006. 


happen: a melody that spins around in your head, your 
inner voice talking to yourself. Other sounds not heard 
aloud are planned: practicing lines for a school play 
or rehearsing a phone number before dialing. Where 
are these sounds processed in the brain? We are aware 
that we actually seem to 'hear' these inner sounds. 
Does that mean that auditory cortex is activated when 



FIGURE 7.43 Illustration of brain areas active for imagined 
sounds. Source : Adapted from Zatorre and Halpern, 2005. 


they are playing despite the fact that there is no actual 
sound? Halpern and colleagues (Zatorre & Halpern, 
2005) have investigated this question using neuroimag¬ 
ing techniques to measure brain activation for imagined 
sounds versus heard sounds. Results (shown in Figure 
7.43) show that non-primary auditory cortex is indeed 
active during imagined - and not heard - sounds. 

A related finding was reported by Jancke and col¬ 
leagues (Bunzeck et al., 2005). These investigators 
wanted to study auditory imagery for environmental 
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Perception vs. Control Imagery vs. Control 



(a) (b) 


FIGURE 7.44 fMRI study of perceived sounds versus imagined sounds. The sounds used in 
this study were neither language nor music, in order to determine the localization of imagined non- 
linguistic or musical sounds. Primary auditory cortex was active during the perception phase of the 
experiment but not during the imagery phase. Source : Adapted from Bunzeck et al., 2005. 


sounds. Using fMRI, they recorded neural responses 
to subjects perceiving sounds and imagining those 
sounds. Results are presented in Figure 7.44: primary 
and secondary auditory cortex in both hemispheres 
is active when perceiving sounds (left panel), while 
secondary (and not primary) auditory cortex is active 
when imagining those same sounds (right panel). 

These findings provide compelling evidence that 
imagined sounds activate similar neural regions in 
auditory cortex that are activated when sounds are 
heard. Recall the learning data from Weinberger and 
colleagues showing that sound-based memory and 
learning occurred in auditory cortex. The findings pre¬ 
sented here, while representing only a small propor¬ 
tion of the on going investigation of auditory imagery, 
indicate that similar processes occur in humans as 
well, with imagining and perceiving sounds sharing 
neural territory 

8*0 SUMMARY 

In this chapter, we presented an overview of the com¬ 
plex auditory system, from hearing basics to music 


perception to auditory imagery The advent of neu¬ 
roimaging techniques has provided a wealth of new 
data for understanding the cortical auditory system 
and how it interfaces with other cortical regions. 
While we have made major inroads on understanding 
the puzzle of auditory perception, there is still much 
work to be done. For example, teasing apart neural 
systems that underlie music and speech perception 
is still in the early phases. There are many other key 
questions that are being addressed in the field of audi¬ 
tory brain science. For example, what are the differing 
roles of the left and right hemispheres in speech and 
music perception? 

There is fruitful work in the investigations of 
processing streams in the auditory system and in the 
brain. And while the work in non-human primates 
has informed us greatly about the existence of 'where' 
and 'what' processing streams, these streams may be 
established differently for humans due to the unique 
and important roles of speech and music perception 
in the evolution and development of the human brain. 
The next time an uncalled melody plays inside your 
head, consider the areas that might be activated in 
your brain as you 'hear' your silent song! 



236 


7. HEARING AND SPEECH 


9*0 CHAPTER REVIEW 
9.1 Study questions 

1 What are the basic physical features and psy¬ 
chological aspects of sound? 

2 What are the main parts of the auditory system 
and what are their roles in perception? 

3 Briefly describe some differences between the 
'what' and 'where' processing streams. 

4 What are the basic units of analysis for speech 
perception? 

5 What have new brain imaging techniques pro¬ 
vided us in terms of investigating auditory 
function? 


9*2 Drawing exercise 

We highly recommend drawing and coloring to 

help you remember the physical layout of the brain. 

1 Top panel of Figure 7.45: identify auditory corti¬ 
cal areas that are visible on the lateral aspect of 
the brain. 

2 Bottom panel of Figure 7.45: identify the audi¬ 
tory cortical regions denoted by blue, pink, and 
brown shading. 

9*3 Exploring more 

Suggested reading and web sources: 

Blumstein, S. E. (1997). A perspective on the neurobiology of lan¬ 
guage. Brain and Language, 60(3), 335-346. 

Hickok, G., & Poeppel, D. (2004). Dorsal and ventral streams: A 
framework for understanding aspects of the functional anat¬ 
omy for language. Cognition, 92, 67-99. 

Moore, B. C. (2002). Psychoacoustics of normal and impaired 
hearing. British Medical Bulletin, 63, 121-134. 

Peretz, I., & Zatorre, R. J. (2005). Brain organization for music 
processing. Annual Review of Psychology, 56, 89-114. 



Frontal pole 



FIGURE 7.45 An axial (horizontal) slice of the brain show¬ 
ing the anatomy of auditory cortex in left and right hemispheres. 


NIH website on the auditory cortex <http://www.ncbi.nlm.nih 
.gov/books/bv.fcgi?rid = neurosci.section.919/>. 

NIH website for searching the literature on auditory processes 
Entrez Pubmed - < http://www.ncbi.nlm.nih.gov/entrez/>. 

You can enter names of scientists mentioned in this chapter into 
a search engine like Google. Many have laboratory websites 
that are interesting to explore. <http://www.google.com/>. 
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Millions of items ... are present to my senses which never properly enter into my experience. 

Why? Because they have no interest for me. My experience is what I agree to attend to _ 

Each of us literally chooses , by his ways of attending to things , what sort of a universe he shall 
appear to himself to inhabit. 


William James (1890), The Principles of Psychology 



The central scene of a famous painting by Diego Velazquez (1599-1660) called Las Meninas, or The Maids of Honor, who are the Royal com¬ 
panions of the Infanta (the little girl at the center) the Princess of Spain and the daughter of King Philip IV and his Queen - who are shown 
in the shadowy painting at the back of the room. It is widely believed to be a contemplation of self-consciousness, one of the topics of this 
chapter. Notice that almost everyone in this painting seems to be paying attention to themselves, or to a mirror image of themselves, start¬ 
ing with Velazquez himself (on the left) who is looking into a large mirrror in order to paint the entire scene. We, the viewers, are therefore 
part of the scene, because it is painted from the viewpoint of the audience. The Infanta herself seems to be wondering if she is looking pretty. 
Children develop social self-consciousness around the age of four. It is a profoundly important aspect of the human mind-brain and of our 
social existence. ( Source : Wikimedia. The original painting hangs in the Prado Museum in Madrid.) 
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L0 INTRODUCTION 

We wake up each morning to an ongoing stream of con¬ 
scious events. As we open our eyes, the visual world 
comes alive again. A flow of coherent inner speech also 
begins as we wake up - we start to talk to ourselves. 


Conscious contents begin to be encoded as episodic 
memories, so that we can learn and recall experiences (see 
Chapter 9). By comparison to the waking state, we gen¬ 
erally have poor recall for sleep events, even for dreams 
that occurred just before awakening. Yet we dream for 
90 to 120 minutes each night. Therefore, most dreams 
will never be recalled. 


Cognition, Brain , and Consciousness, edited by B.J. Baars and N. M. Gage 
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As we come to consciousness each morning we 
remember our most important current goals. That means 
we can exercise voluntary attention over the informa¬ 
tion that will come to our consciousness. As William 
James wrote (above), 'My experience is what I agree to 
attend to.... Each of us literally chooses, by his ways 
of attending to things, what sort of a universe he shall 
appear to himself to inhabit'. James does not explain 
spontaneous attention - the things we pay attention to 
even though they are not voluntarily brought to mind. 
But he explains how over time, the things we choose to 
pay attention to come to dominate our conscious wak¬ 
ing periods. 

As we awaken we can control our skeletal muscles 
again - unlike during the near-paralysis of sleep - so 
that we can stand up, scan the world, go hunting and 
gathering, and talk with each other. Each of those capac¬ 
ities recruits a different network of brain events that can 
only be learned, mobilized, and controlled in the wak¬ 
ing state. In the waking state we have a vast repertoire 
of abilities that are not available in other states. 

Our functional diagram (Figure 8.1) shows the con¬ 
cept of 'attention' as an arrow going from attentional 


control regions to cognitive processes that are enhanced 
by attention. Voluntary attention is shown as an arrow 
running from the central executive (roughly, the fron¬ 
tal lobes) to brain activities that are enhanced by atten¬ 
tion. Sensory cortex can also be enhanced by events 
that are not under voluntary control. If someone yells 
out loud, a large dog barks unexpectedly from a few 
feet away, or a truck looms into your field of vision, 
attention will be stimulus driven. Many biologically 
significant events trigger attention 'bottom up', such 
as the smell of food when you are hungry. We spon¬ 
taneously select personally significant stimuli as well, 
like the sounds of our own names. We can symbolize 
stimulus-driven attention with an arrow coming from 
the sensory boxes of the diagram to suggest that some 
stimuli are inherently attention-capturing. 

The sensory regions of the posterior cortex are espe¬ 
cially sensitive to attentional enhancement. However, 
the motor cortex, Broca's area, and semantic memories 
can also be selectively attended. 

If we take conscious experiences in many cases to 
be the results of selective attention, that can also be 
readily expressed in the diagram. Although there is 


A functional framework. 
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FIGURE 8.1 A functional framework for attention and conscious events. How consciousness and atten¬ 
tion appear in the functional framework. Notice that attention can be under voluntary control by the central 
executive, since people can be asked to pay attention to a great variety of events. However, 'bottom up' atten¬ 
tional capture occurs when sensory events are intense, surprising, or significant enough to make us pay atten¬ 
tion. Paying attention usually leads to specific conscious experiences. Source: Bernard J. Baars, 2009. 
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scientific debate as to whether all conscious experi¬ 
ences result from attentional selection, no one doubts 
that many of them do. Everyone who teaches has 
called the attention of the class to conscious events; it 
is how we begin any group meeting. Human and ani¬ 
mal experiments do much the same thing. The link 
between attention and consciousness is an intimate 
one, a big part of our everyday psychology. 

With careful studies we can separate the phenomena 
of attention and consciousness. To focus on conscious 
events 'as such', we typically study them experimen¬ 
tally in contrast with closely matched unconscious 


events, as we have seen in previous chapters (see 
Chapters 1, 3, 6, and 7). By contrast, experiments on 
attention typically ask subjects to select one of two 
alternative stimuli. 'Attention' is therefore concerned 
with the process of selection and consciousness, with 
reportable experiences themselves. 

Some key questions for cognitive neuroscience 
are: What is distinctive about conscious events in the 
brain? What does it really mean for someone to be 
conscious? And how does the brain basis of atten¬ 
tional selection relate to our private, conscious experi¬ 
ences of the world? 


FRONTIERS OF COGNITIVE NEUROSCIENCE 


Consciousness 



FIGURE 8.2 Christof Koch (left) is a well-known neurobiologist and researcher on the brain basis of consciousness. Koch worked 
for many years with Francis H.C. Crick (right), the Nobel Laureate and co-discoverer of DNA. Crick (1916-2004) devoted the last dec¬ 
ades of his life to this topic and inspired other scientists to follow his lead. 


How can we study consciousness? How can the relation¬ 
ship between subjective experiences, what philosophers 
refer to as qualia (the elements making up any one experi¬ 
ence, whether it is the experience of seeing red, smelling 
cow dung, or being angry), and the brain be understood in 
an objective, quantitative, and empirical manner? Initially, 
there was a lot of resistance to the idea that one could study 
consciousness - whether in humans or in related animals, 
such as mice or monkeys - scientifically. "Leave that work 
to retired Nobel laureates, philosophers, artists, literary 
people and the pyramid crowd"! was a common reaction. 
However, consciousness is a brute fact of nature. And it is, 
of course, the only way I - or anybody else - experience 
the universe and the only way I know that I exist. This is 
the genius behind the most famous deduction in Western 
thought, Descartes' Je pense , done je suis. If we desire a sci¬ 
entific comprehensive view of the universe and everything 


in it, then consciousness must figure prominently in such a 
final description of everything there is. And over the past 
twenty years, we believe that we have made significant 
empirical progress in an inchoate science of consciousness. 
The approach that Francis Crick and I championed since 
our first paper on the topic in 1990, focusing for now on 
the Neuronal Correlates of Consciousness , the minimal neu¬ 
ronal conditions jointly necessary for any one specific con¬ 
scious sensation, has borne ample fruits. 

But when we have finally identified the elusive NCC, 
say for adult human visual perception, we will need to 
understand why this particular NCC does give rise to 
consciousness, and why not that one. And we will need 
to understand which nonhuman organisms are conscious. 
Although this is easy for closely related species, such as 
monkeys and mice, and probably for all mammals, this 
becomes much more tricky for cases that deviate from 
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this standard. Is a fetus conscious? What about one of 
the thousands of persistent vegetative state patients? A 
bee? A squid? To answer these and other questions in a 
definite manner we need a fundamental theory of con¬ 
sciousness. And the idiom of such a theory needs to be 
the mathematic of information theory, since information 
can express causal relationships among the many parts of 
any system independent of the substrate of this system, 
whether spiking neurons or silicon transistors. 

In my eyes, the only plausible candidate for a funda¬ 
mental theory of consciousness is the one by Giulio Tononi. 
Called the Integrated Theory of (ITT) Consciousness, it 
provides an exciting new way to study consciousness 
using a rigorous scientific approach (Tononi & Edelman, 
1998; Balduzzi & Tononi, 2008; Tononi, 2008). 

The ITT framework is built on the notion that con¬ 
sciousness is a consequence of systems that have both a 
large amount of differentiated information that is also 
highly integrated. In this context, integrated information 
means information that is accessible to the entire system 
as a whole. An iPhone, for example, has 16 Gbytes of 
storage space. This represents around 16 billion discrete 
distinct states. But they are not at all integrated. To the 
iPhone, each pixel in each image is identical to any other 
information, say, in its calendar. It does not realize, for 
example, that the various pictures of a young girl on my 
iPhone morphing into a self-assured woman is that of my 
daughter whose entry is in my phone book, and that she 
is related to the smiling confident man - her brother - in 
the next batch of images. For the current generation of 
iPhones (and other computers), all these pixels are equally 
meaningless (it does not even realize that images are 


inherently 2-D). ITT provides a precise way to calculate 
the size of the conscious repertoire, the state of conscious¬ 
ness as it were, using a number, measured in bit, called <1>. 

Tononi's ITT provides a critical tool to ground con¬ 
sciousness in the natural world in a quantitative manner. 
According to Tononi, the amount of integrated informa¬ 
tion that you or I have in our brains corresponds to our 
repertoire of possible conscious states. Many fascinating 
questions are raised by such a theory. One is particularly 
fascinating from the point of designing machines (I'm not 
only a professor of biology but also one of engineering). Can 
artifacts made out of silicon, copper wire, and steel even be 
sentient? Can they too experience the world? 111 clearly 
argues yes, provided they have both highly differentiated as 
well as integrated information (Koch & Tononi, 2008). 

For more information about the neurobiological 
aspects of consciousness, see my book. The Quest for 
Consciousness: A Neurobiological Approach. The research 
we carry out in my laboratory is described in detail at 
http://www.klab.caltech.edu/. 
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1.1 Waking cognition is consciously mediated 

The waking state supports an endless set of adaptive 
functions. During waking consciousness we learn new 
actions, feelings, words, ideas, skills, goals, and even new 
kinds of sensory events, like a new way of understand¬ 
ing music or visual art; each one involves distinctive pat¬ 
terns of brain activities. The set of waking state functions 
is therefore an open set, one that can keep expanding as 
long as our brain is alive. The most basic ones are dis¬ 
cussed throughout this book: vision, audition, memory, 
emotions, executive control, social cognition, and so on. 

Each waking task has both conscious and uncon¬ 
scious components. For example, the lower ventral 
stream of vision yields reportable conscious object 
representations, like visual coffee cups or a kitchen chair. 
The upper dorsal stream of the visual brain represents 
body space and controls actions like manual reaching. 


but its contents are not reportable as conscious (Milner 
and Goodale, 2007). Yet both the dorsal and ventral 
streams require the state of waking consciousness to 
work. Almost all cognitive tasks we know take place 
during the waking state and have both conscious (report- 
able) and unconscious (nonreportable) components. 

Most cognitive tasks we know are therefore con¬ 
sciously mediated. Working memory, for example, has 
both conscious and unconscious components. If you 
rehearse seven numbers you will notice that some 
are conscious at any moment, but others are not. The 
instructions to rehearse and remember are obviously 
conscious and so is the set of items as you see or hear 
them. But we are not aware of the nonrehearsed items at 
any moment, of the important role of the basal ganglia 
in controlling inner speech, or of the automatic (habit¬ 
ual) components of any task. There are no completely 
conscious cognitive tasks, as far as we know, and there 
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may be no completely unconscious ones (Baars and 
Franklin, 2003; Prof. Stan Franklin has suggested the 
term 'consciously mediated' for cognitive tasks that 
have a conscious but are otherwise unconscious). 

The basal ganglia and cerebellum are very large 
brain structures that are believed to function without 
supporting moment-to-moment conscious contents, 
even in the waking state. The cerebellum can actually 
be lesioned on both sides, and people and animals will 
continue to behave much as before but without the 
ability to control fine motor movements. In humans 
those structures have many other cognitive roles but 
without leading directly to conscious contents. 

What can we do completely unconsciously? We 
still do not know the answer, because it is difficult to 
do careful studies on sleepwalking, sleep movement 
disorders, epileptic 'automatic behaviors', and other 
'zombie states' (Crick and Koch, 2003). There are many 
reports about automatic behaviors from individu¬ 
als with sleep disorders and epilepsy. To verify those 
reports we need brain recordings that are difficult to 
obtain in freely moving people. It is also possible that 
epileptic behavioral automatisms, for example, reflect 
momentary conscious 'flashes' that are known to exist 
(Kranczioch et ah, 2006). It is therefore hard to test 
whether there are complex but entirely unconscious 
behaviors, in part because we simply do not know the 
distinctive brain correlates of consciousness as yet (but 
see Gaillard et ah, 2009; Revonsuo, 2008). We do not 
know yet what difference enables consciousness of the 
ventral but not the dorsal stream of the visual cortex. 
There are ongoing efforts to make progress on those 
questions, however (Laureys and Tononi, 2008). 


1.2 The need for timely waking, sleep, and 
dreaming 

Chronically sleep-deprived rats die after only three 
weeks. We know, therefore, that sleep is needed for 
survival in mammals, but its exact functions are still 
unclear. Sleeping animals are more vulnerable to preda¬ 
tors. Our nightly eight hours of loss of consciousness of 
our environment must therefore have some compensa¬ 
tory advantages. We do know that sleep and dreaming 
seem to enable memory consolidation, converting new 
and unstable memories into lasting ones. Sleep, dream¬ 
ing, and waking are complex states, with long evolu¬ 
tionary histories. Like other major adaptations they 
have probably gathered many biological functions. A 
large number of genes are known to be expressed in 
sleep, and many genes change their expression due to 



FIGURE 8.3 Three dimensions of waking, sleep, and dreaming. 
Many state-related phenomena can be described in three dimensions 
(Hobson et al., 2000; Giocomo and Hasselmo, 2007). "Fast cortical 
activation' refers to widespread, fast, irregular, and low-amplitude 
activity in the EEG. 'Input sources' are either internal, as in dream¬ 
ing, or external, as in waking. 'Neuromodulation' refers to four 
widespread neurochemicals that are turned on and off by basal brain 
nuclei. They are acetylcholine (ACh), norepinephrine (NE), serotonin 
(5-HT), and glutamate (Glut). These molecules play vital roles in per¬ 
ception, action control, and learning. SWS refers to slow-wave sleep, 
marked by delta waves in the raw EEG. SWS has the highest arousal 
threshold of the circadian cycle. Source : Adapted from Hobson, 2000. 

sleep deprivation. Scores of biological parameters seem 
to be turned off and on by sleep and its loss. 

Sleep duration appears to be under homeostatic con¬ 
trol, since we tend to make up for lost sleep by sleeping 
longer. Sleep-deprived individuals show 'microsleeps', 
moments of 'dropping off', that can seriously interfere 
with actions like driving. You can observe moments of 
dropping off just by studying late into the night. When 
your brain rapidly switches from waking to sleep, 
state-modulating brain and spinal chemicals switch 
quite quickly (Figure 8.3). In a second or two, your neck 
muscles can lose tone, and your head will tend to drop 
forward. Spinal and cranial motor nerves switch to 
inhibition, and it's time to get some rest. We have some 
cortical control over staying awake, but eventually the 
biological clock will win out. 

Surprisingly, a single night of sleep deprivation can 
also improve depression, even when other treatments 
have failed. Staying up for one night therefore has been 
suggested as a safe, inexpensive, and effective treat¬ 
ment for depression. But chronic sleep deprivation is 
stressful and degrades normal conscious functioning. 

The sleep-waking cycle is controlled by a biologi¬ 
cal clock, triggered in part by daylight onset and off¬ 
set. A small group of retinal light receptors detect light 
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onset, signaling the suprachiasmatic nucleus, the pineal 
gland, hypothalamus, and deep brain nuclei, in order to 
release state-specific molecules. The major neuromodu¬ 
lators have state-specific actions (Figure 8.3). Changes 
in serotonin, norepinephrine and acetylcholine have 
been well described to trigger changes in the major 
brain states (see Chapter 16). 

13 Global rhythms of circadian states 

At the moment of awakening from sleep, billions of 
neurons change from the slow, unified 'buzz-pause' 
of slow-wave sleep to much more differentiated and 
task-specific signaling. The difference between sleep 
and waking is not just behavioral and subjective. 
Figure 8.4 shows the basic patterns we find in the 
'raw' (unanalyzed) EEG in the three basic states. 

Although scalp-recorded EEG is very useful, it is 
relatively crude, since it shows the additive results of 
billions of fast electrical signals from huge cell popula¬ 
tions, filtered through dense layers of tissue and bone. 
Much effort has been devoted to finding out the brain 
basis of these states by using depth electrodes and 
brain imaging. Much of what we know about the thal¬ 
amocortical system comes from studies in other mam¬ 
mals and in human medical patients. All mammals 
share the basic thalamocortical system. The raw (unan¬ 
alyzed) EEG seems relatively crude, but even surface 
EEG shows fundamentally important information 
about sleep, waking, and some disorders like epilepsy. 

Figure 8.4 shows the difference in surface EEG 
between waking, sleep, and REM dreaming. Each 
EEG state also has typical subjective experiences. 
Surprisingly, slow-wave sleep shows some mental 
activity as well; however, deep sleep is statistically less 
likely to yield reports of mental activity. Deep sleep is 
the least conscious state of the normal daily cycle, as 
measured by behavioral arousability - the likelihood 
that an animal will sit up, orient its receptors, and 
show waking EEG in response to a stimulus. 

The standard surface EEG is generated by near- 
to-the-scalp branches of cortical neurons; these are 
typically the dendrites at the apex (top) of pyramidal 
cells - pyramid-shaped neurons with long axons that 
point downward from the scalp. Their long axons 
connect to other parts of the cortex or the thalamus. 
Pyramidal cells are excitatory, using glutamate as their 
major neurotransmitter, but they are surrounded by 
small inhibitory interneurons that use GABA as their 
neurotransmitter. Excitatory and inhibitory neurons 
work together in the same local patch of cortex. 
Cortical neurons are stacked in minicolumns of six 
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FIGURE 8.4 Surface EEG of the major states. The brain generates 
an electromagnetic field that can be recorded on the scalp. Each trace 
represents field oscillations resulting from billions of small neuronal 
voltages near the surface of the cortex, filtered through layers of tissue. 
The raw EEG reflects a complex mix of many different waves, but each 
state shows dominant waveforms that are quite distinctive. Notice the 
similarity between waking EEG (top) and REM dreaming (bottom). 
Judging by brain electrical activity, REM dreaming looks like a con¬ 
scious state - which is indeed what we experience during dreams. As 
we get drowsy and go into deeper stages of sleep (Stages 1-4), sleep 
spindles, K complexes, and slow delta waves appear. (Delta is defined 
as less than 3 Hz.) More recently, slow oscillations have been studied, 
going down to .01 Hz. In deep sleep, mainly delta waves are visible, 
reflecting the simultaneous "buzz' and "pause' of billions of neurons 
firing and pausing together. However, it is now known that very slow 
oscillations occur even during the waking state, cps = cycles per sec¬ 
ond; Source : Squire, 2008. 


layers (see Chapter 5), which interact constantly with 
thalamic cells, so that in the surface EEG we are seeing 
the massive summed oscillations of the thalamocorti¬ 
cal system as a whole. That system is the brain basis 
for states of consciousness. 

It has been difficult to get 'clean' brain signals using 
scalp EEG, although there has been marked progress 
over the years. As you know from Chapter 4, MEG 
allows for better spatial localization than surface EEG, 
though it picks up slightly different sources. Local 
field potentials can be recorded throughout the brain 
with depth electrodes. Nevertheless, EEG is the most 
widely used brain recording technique because it 
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FIGURE 8.5 Slow hypersynchrony in epileptic loss of consciousness. The surface EEG of a 7-year-old 
girl during an epileptic seizure with loss of consciousness. Epileptic loss of consciousness shows slow, hyper- 
synchronized global EEG similar to deep sleep but characteristically more jagged. Other unconscious states, 
like coma and general anesthesia, also show slow, high-voltage, and synchronized waves spreading over the 
cortex, suggesting that synchrony by itself is not the marker of consciousness. Source : Blemenfled, 2005. 


is relatively inexpensive, picks up vitally important 
medical phenomena (like epilepsy and sleep disor¬ 
ders), and careful mathematical analysis helps to 
'clean up' the signal. 

EEG resembles the kind of information space aliens 
might pick up from radio signals coming from the earth 
without understanding much human speech and with 
only a vague idea of what human beings might be say¬ 
ing. Nevertheless, there has been marked progress in 
understanding (e.g. Buzsaki, 2002). Intracranial record¬ 
ing is more accurate and produces much less noise, with 
the major drawback being its invasive nature. By far, 
the greatest number of depth recordings has been per¬ 
formed in other animals, but human intracranial EEG 
(iEEG) has now emerged as a major source of scientific 
evidence. iEEG goes back to the pioneer neurosurgeon 
Wilder Penfield some 50 years ago. The signal strength 
inside the brain is up to 1000 greater than on the scalp, 
and a wider range of frequencies can be recorded. In 
addition, iEEG is free of some experimental artifacts, 
such as the stray voltages coming from eye movements 
and scalp muscles. Human iEEG is performed only in 
medically justified circumstances, but it has added a 
great deal to our knowledge, as we will see. 

Large brain regions outside of the thalamocortical 
system may have little direct effect on conscious states. 
For example, humans and animals with bilateral loss 


to the cerebellum show impaired fine motor control 
(finger and hand movements), but they retain waking 
consciousness of the world and of themselves. More 
recent research shows that the cerebellum does have 
cognitive functions, probably by interacting with cor¬ 
tex. Since the cerebellum contains about as many neu¬ 
rons as the cortex itself, this basic fact seems to show 
that consciousness is not just a result of a large number 
of neurons. Something else is required. 

Edelman and Tononi (2000) have proposed that it 
is the high interactivity of the cortex and thalamus that 
enables it to support a vast repertoire of consciously 
mediated tasks and experiences. Figures 8.5 and 8.6 
show the anatomical basis of that highly interactive 
and flexible capacity. By comparison, the cerebellum is 
much more compartmentalized, with separate groups 
of neurons that seem to operate in parallel with each 
other. The World Wide Web is a natural analogy for 
the thalamocortical system, since any computer on the 
Web can communicate with any other. 

Deep sleep is the most unconscious state of the 
daily cycle, marked by slow, high-voltage waveforms, 
involving coordinated 'Up States' and 'Down States' 
among very large cell populations. The EEG of deep 
sleep has some striking similarities to epileptic loss 
of consciousness, as well as comatose conditions and 
general anesthesia (Figure 8.5). 
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FIGURE 8.6 Three neurons in a 
thalamocortical loop. The basic rhythmic 
'pump' of the brain. Waking, sleep, and 
dreaming are driven by thalamocorti¬ 
cal oscillations. Thalamic nuclei interact 
closely with corresponding regions of 
cortex. This core brain is shared by other 
mammals and birds. Source : Adapted from 
Steriade, 2006. 


Epilepsy may impair consciousness due to hyper¬ 
synchrony of vast numbers of cortical and thalamic 
neurons. Figure 8.5 shows an EEG taken of an epileptic 
seizure in a 7-year-old girl during loss of consciousness 
(Blumenfeld, 2004). Notice the slow, high-amplitude 
waveforms, especially in her frontal lobes. These hyper- 
synchronous waves often spread by 'recruiting' other 
parts of cortex. fMRI and PET show a drop in metabolic 
activity during seizures in the frontal and parietal lobes. 
Notice the rough similarity between natural slow-wave 
sleep (Figure 8.4) and loss of consciousness in epilepsy. 
Both of the EEGs show slow, synchronous activity in 
large parts of the thalamocortical core. However, the epi¬ 
leptic waves are much 'spikier' than deep sleep delta. 1 

There is an apparent paradox in global hyper¬ 
synchrony in the EEG in unconscious states: On the 
one hand, synchronized brain rhythms allow widely 
separated regions of the brain to work together. On the 
other, the electrical storms of global hypersynchrony 


disrupt ordinary brain functions. It appears that nor¬ 
mal survival activities cannot be carried out during 
global, hyper synchronous episodes. 

One answer is that normal cognition requires selec¬ 
tive , local synchrony among brain regions, and that the 
speed of working synchrony is faster than the —1Hz 
waves of deep sleep and epileptic seizures. Waking 
tasks use theta, alpha, beta, and gamma synchrony, 
covering a range of frequencies from about 4 to 200 Hz 
or even higher. Even 1000 Hz bursts have been 
observed during waking. Waking oscillations are 
highly patterned and differentiated, so that synchrony, 
desynchrony, and aperiodic 'one-shot' waveforms 
constantly appear and disappear (Tononi, 2004). 

Waking consciousness resembles the flow of traf¬ 
fic in a busy city. At night, street traffic might dwindle 
to a few cars, but at rush hour, vehicles are constantly 
going from any location to any other. It is that abil¬ 
ity to go anywhere, guided by local goals, that makes 


1 Coma patients, as well as patients under general anesthesia, may also show high and slow EEG. It seems that unconscious states commonly 
show high, slow, coordinated waves. In the worst coma cases, however, EEG will drop to very low voltages. Medical brain death is often 
defined as zero voltage in the EEG. However, recent studies show that conventional scalp EEG may not pick up residual brain activity in cases 
of coma. Medical researchers have therefore proposed a new diagnostic category, the "Minimally Conscious State' (MCS), which may mimic 
true coma but still allows for conscious moments. A high percentage of patients who were previously diagnosed with irreversible coma are 
now believed to be in MCS, with intermittent conscious episodes (Laureys and Tononi, 2008). 
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FIGURE 8.7 EEG and single-unit activity in waking and deep sleep. Conventional EEG meas¬ 
ures the brain's electrical field over the scalp. Each EEG trace in the figure is a complex sum of under¬ 
lying neuronal activity in the upper layer of cortex. Slow-wave sleep EEG reflects the simultaneous 
'buzz' and 'pause' of billions of neurons, also called 'up' and 'down' states. Because waking-state 
neurons do not fire and pause synchronously, their voltages do not add up to large waves. The wak¬ 
ing EEG looks small, irregular, and faster than slow-wave sleep. It is believed that waking (and REM 
dreaming) therefore involves more differentiated information processing in much the same way that 
a stadium full of talking people serves to process more information than the same people all chant¬ 
ing in unison. The unison chanting is largely redundant (you can predict the crowd chants from just 
one person) so the information content is lower (see Section 6.2). Adapted from Steriade, 2006. 


vehicles useful. Similarly, waking tasks serve a huge 
variety of functions with great flexibility In contrast, 
in slow EEG hypersynchrony all the traffic lights seem 
to turn green for a few seconds, then red, then green 
again, over and over again. The flow of traffic there¬ 
fore is forced to screech to a halt in unison, then start 
again, stop again, and so on. The normal, flexible flow 
of brain signaling is constantly disrupted. 

L4 States of consciousness have characteristic 
thalamocortical activity 

Figure 8.6 shows the basic unit of the thalamocortical 
system, consisting of three connected neurons in cor¬ 
tex, the thalamus, and the reticular nucleus of the thal¬ 
amus. The cortical neuron in the figure is a pyramidal 
cell, named after the pyramid-shaped body of the cell. 
These are the long-distance excitatory neurons of the 
cortex, often surrounded by smaller cells. Keep in mind 
that the thalamus is considered to be the 'gateway to 
cortex' and that most thalamic nuclei have a corre¬ 
sponding region of cortex with which they interact. 
Some thalamic nuclei, like the LGN (see Chapter 6), 
involve relay neurons from sensory to cortical regions. 
Others are corticocortical, allowing attentional systems 


of the parietal and frontal lobe to enhance sensory 
processing in posterior cortex (via the pulvinar and 
mediodorsal nuclei of the thalamus). 

Together the neurons in Figure 8.6 form a circuit 
that determines the major states of the circadian cycle. 
A fourth cell deep in the brain can turn this circuit 
on and off by squirting neuromodulating chemicals 
throughout large parts of the forebrain. There are 
many millions of such circuits, yet the basic simplicity 
of this thalamocortical loop is striking (Figure 8.8). 

We can think of such a loop as an oscillatory circuit 
consisting of three oscillating units. Like the children 
on two playground swings that are suspended from 
a single crossbar (see Chapter 3), each unit can be 
treated as an oscillator. But the swing set as a whole 
is also an oscillating system. Single neurons are fast¬ 
charging and fast-discharging electrical wave gen¬ 
erators. Circuits of neurons oscillate in more complex 
patterns. The thalamocortical loop of Figure 8.6 can 
be thought of as a fundamental rhythmic pump of the 
brain, operating differently in the global brain states 
of waking, slow-wave sleep, and REM dreaming. 

Figure 8.7 shows how during waking, millions of 
individual neurons are firing largely independently of 
each other. Even though populations of neurons fire in 
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synchrony during the waking state, their interactions 
are relatively fast (perhaps 4-200 Hz) and local; fast 
synchrony appears to be task related, and it alters as 
mental activities proceed. As a result, neuronal firing 
in waking does not add up to large, global waveforms. 


Thalamus 
cortical activation 
sleep spindle 
EEG synchronization 



Hypothalamus 
sleep/wake switch 

SCN 


Brainstem 
ascending cortical activation 


Circadian clock REM/SWS switch 


FIGURE 8.8 Brain state switches. The light cycle is picked 
up by special retinal receptors, which connect to the SCN (supra- 
chiasmatic nucleus, located just about the X-shaped 'chiasm' of the 
crossing optical nerves). SCN is part of a circadian circuit that runs 
many brain and bodily functions. State changes are triggered by 
small brainstem and basal brain nuclei that send out neuromodulat- 
ing fibers (see Chapter 16), which change the firing pattern of many 
billions of other neurons. The brainstem ascending cortical system 
participates in switching from slow-wave sleep (SWS) to REM. The 
hypothalamus, the endocrine center of the brain, is involved in the 
sleep-wake switch. The thalamus connects to all parts of the cortex 
and is a way station of almost all sensory nerves on their way to 
cortex. In SWS it generates sleep spindles (Figure 8.6). Together, 
these regions control the daily cycle of major brain states. They are 
the traffic control system for tens of billions of other neurons and 
trillions of synaptic connections. Source : Bacon et al., 2007. 


unlike SWS (Figure 8.7, bottom). In SWS, massive 
numbers of neurons in the thalamocortical system fire 
simultaneous bursts and pauses about every other sec¬ 
ond, adding up to slow EEG waves in the global EEG. 
Such delta waves are conventionally thought to range 
from 0.5 to 3 Hz, but they have recently been found to 
extend into the slow oscillation range (0.01-0.5 Hz). 

Another useful analogy is that SWS resembles a 
large football stadium in which all the fans chant and 
pause in unison every other second or so. When they 
shout together, the summed amplitude of the sound is 
very high. When they pause together, the total sound 
is very low. But when the audience members are just 
chatting with their neighbors, the sound is irregular, 
relatively low in amplitude, and much more complex. 
From this point of view, it is the waking state that car¬ 
ries the greatest amount of useful information (Tononi, 
2004; see Section 7.0). 

Current evidence based on these sources indicates a 
vital role for synchronized and interacting brain oscil¬ 
lations. They are believed to enable: 

• Attention, perception, and conscious contents 

• Learning, neural plasticity, and memory retrieval 

• Long-range coordination between brain regions, 
including sensorimotor interaction 

(Canolty et al., 2006). 

Brain waves often interact with each other, with 
slower rhythms tending to group faster ones. Interactions 
are often transient and related to specific tasks, as we will 
see. Both synchrony and desynchrony can play a role. 


1.5 The thalamocortical core: massively 
interlinked and very active 

It is easiest to think about the wiring of thalamus 
and cortex in terms of simple circuits, as described in 


Berger bands Gamma Ripple 

Fast ripple 


DC Infra-slow Slow Fast Multi-unit and unit 

^ ^ ■....- --I___. . - - i - ■ ■ —* - - - ■ - - ^ 

icr 2 icr 1 io° io 1 io 2 io 3 io 4 

Frequency (Hz) 

FIGURE 8.9 Brain rhythms range over several orders of magnitude. The wide range 
of functional brain oscillations. On the left, very slow activity appears as a rise and ebb of 
the voltage baseline. Very slow oscillations occur at <lHz, followed by theta, alpha, beta, 
and gamma. These labels are somewhat arbitrary. Very high-frequency events have been 
called 'ripples'. On the right side, single neurons and multiunit groupings show spikes 
up to 10 KHz. The range from 80 to 150 Hz is sometimes called the 'high gamma' range. 
Source: Bazhenov and Timofeev, 2006. 
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Figure 8.10 - just as it is easiest to think about auto¬ 
mobile traffic in a large city in terms of single car trips 
from point A to point B. But it's also important to keep 
in mind the massive size of the whole metropolitan 
area - the thalamocortical system and its constant flow 
of traffic, with many 'signal trips' occurring at the 
same time. Izhikevich and coauthors have run large- 
scale simulation models of the system, which help us 
to understand how it works (Izhikevich et al, 2004, 
2008). The figures in this section show various aspects 
of the active core brain. 

Most of the visible mass of the inner brain consists 
of white-covered neural fiber tracts, the myelinated 
neurons and their axons. These high-traffic highways 
have now been shown to follow the mathematics of 
'small world networks', like the World Wide Web, 
which allows for the most efficient flow of signal traf¬ 
fic from any starting point in the brain to any desti¬ 
nation (Achard et al., 2007). Basically, a small-world 


network consists of high-traffic highways and hubs, 
along with numerous small streets and alleys for 
much more limited local traffic. Many networks in 
nature follow such an optimally efficient distribution 
of high-traffic and low-traffic links. 

Figures 8.10A and 8.10B show the anatomical basis 
of the massive superhighway networks of the brain. 
The first figure shows the thalamocortical system in the 
left hemisphere, a vast number of two-way streets con¬ 
necting cells in the left thalamus to those in left half of 
cortex. It is the thalamocortical system that we can see 
changing large-scale state activity in the major daily 
states of mind and brain - waking, sleeping, and REM 
dreams. These states are controlled by small nuclei in 
the basal brain, as pointed out earlier, which direct 
'sprays' of neuromodulating axons to large regions of 
the brain including the cortex. 

The second figure shows the cortico-cortical high¬ 
way system, in which the signal traffic flows from one 



FIGURE 8.10A Thalamocortical fiber tracts. The massive size of axonal fiber tracts between 
the thalamus and the outer layers of cortex. Since axons coming from each cell body are wrapped in 
glial cells that are filled with lipid molecules, they appear white to the naked eye, and therefore were 
called the "white matter' by traditional anatomists. These tracts are artificially colored in this compu¬ 
ter-generated graphic, to show the different major pathways. All the fiber tracts seem to emerge from 
the left thalamus from the point of view that is shown here, but at least equal numbers of axons travel 
from cell bodies in cortex to the corresponding nucleus of the left thalamus. Traffic is always two-way 
or 'reentrant' in the thalamocortical system, a fundamental fact that requires explanation. Notice that 
there is no cross-hemisphere traffic in this image. The next figure shows equally massive cortico-cor¬ 
tical highways connecting the hemispheres laterally, mostly flowing across the corpus callosum. As 
Chapter 3 points out, it is possible to lose an entire hemisphere without losing consciousness, as long 
as the second hemisphere is spared along with the brainstem. Source: Izhikevich & Edelman, 2008. 




250 


8. CONSCIOUSNESS AND ATTENTION 



Cortico-cortical fiber tracts run inside of each 
hemisphere, and link corresponding regions of the 
hemispheres across the corpus callosum. 


FIGURE 8.1 OB Corticocortical fiber tracts. The 
massive interconnectivity of the cortex. Like the World 
Wide Web, the cortex shows the mathematics of 'small- 
world connectivity', with highly traveled highways and 
traffic hubs that carry most of the signal traffic, along 
with numerous local and low-traffic streets and alleys. 
Most long-distance signals make use of the superhigh¬ 
ways, the most efficient way for a vast and dense net¬ 
work to operate. Notice that the two hemispheres are 
extensively cross-connected, mostly across the white 
fiber bridge of the corpus callosum. Source : Human 
Brain Mapping Meeting, 2005. 


point in cortex to another one. There are basically two 
kinds of cortico-cortical links, either within the same 
hemisphere, or from one hemisphere to the oppo¬ 
site one. The two hemispheres have an elegant par¬ 
allelism in their connectivity, so that corresponding 
points from back to front of each hemisphere connect 
to the same point on the other side. Although there 
are important functional differences between the two 
halves of the cortex, such as in the production of lan¬ 
guage (which is usually supported by the left side), 
the most striking feature from an anatomical point of 
view is the great symmetry of the two hemispheres. 

There are other major traffic systems in the brain, 
notably the corticospinal tracts, which carry information 
to and from the body via the spinal cord, and the cra¬ 
nial sensory and motor nerves. The visual and auditory 
systems discussed in previous chapters run through 
cranial nerves, whereas the somatosensory and somato¬ 
motor systems run through the spinal cord. However, 
we only need to think about the largest superhighways 
to get a good sense of the vastness of the metropolitan 
map of the brain and its constant traffic flow. 

Keep in mind also that the thalamus is the major input 
hub for the cortex, and also the major cortex-to-cortex 
traffic hub, like a large airport that might serve both 
domestic and international traffic. However, the basal 
ganglia operate as a major output hub, for motor con¬ 
trol and executive functions. The brain has multiple 
hubs, just as it has multiple superhighways. It helps 
to focus on the biggest ones because the basic princi¬ 
ples are generally the same. But mammalian evolu¬ 
tion has had 200 million years to grow the thalamus 


and cortex, on top of the more ancient structures 
like the brain stem, the spinal cord, and the olfactory 
brain. There is a great deal of redundancy in the sys¬ 
tem, just as there are alternative traffic routes in any 
large metropolitan area. 

1.6 Maps and rhythms 

Brain rhythms are much like timers and clocks. They 
support temporal coding in the brain. We also have dis¬ 
cussed topographical maps in vision and other senses, 
which provide spatial codes. We can therefore observe 
both temporal and spatial coding of brain events, pro¬ 
viding a very rich vocabulary of coding possibilities 
for the brain. 

The retina can be thought of as a flat pixel map of the 
visual field (see Chapter 6). When light rays reflected 
from a coffee cup converge on the fovea - the densest 
spot of light receptors in the retina - that light pattern 
is believed to echo through multiple visuotopic maps in 
the visual brain. There are perhaps a dozen different 
visuotopic maps, each representing the visual field at a 
different level of analysis. We can think of those maps 
as echoing each other, beginning with the retina, then 
the visual thalamus and the primary projection area 
for vision. Area VI. Chapter 6 points out that the vis¬ 
ual cortex divides into two separate streams, called the 
dorsal (upper) and ventral (lower) stream. For our pur¬ 
poses, however, we can simply think of the visual brain 
as a stack of maps. This will become especially impor¬ 
tant when we consider the topic of attention, later (see 
Figures 8.11 and 8.31). 
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FIGURE 8.IOC Spontaneous self-organization of cell assemblies in cortex. A large-scale 
active model of massive numbers of neurons and synapses in the cortex and thalamus. On 
the left, just 10% of the cell assemblies that emerged spontaneously in a large-scale model of 
100 000 cortical neurons and 8.5 billion synaptic contacts. Eighty percent of synapses were 
excitatory, with 20% inhibitory links. Neuronal firing was simulated with millisecond preci¬ 
sion. Neurons were connected 'reentrantly', with the two-way flow of activity that is typical 
of thalamocortical signaling. Axonal spikes take a brief time to propagate, depending on the 
length of the axon. This propagation delay was built into the model, and the Hebbian rule was 
applied that 'neurons that fire together, wire together' (Chapter 3). The modeled cell assem¬ 
blies showed self-organization. Stable cell assemblies emerged spontaneously (marked with 
colors on the left side) along with regular cell population rhythms and time-locked activity. 
The model did not simulate external sensory input or motor output, but even purely internal 
signal traffic caused realistic brain-like patterns of activity to emerge spontaneously. Source : 
Izhikevich, Gaily, and Edelman, 2004. 
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FIGURE 8.11 The cortex as a set of 
topographical maps. Cortical representations 
of vision, the body senses, motor output 
regions, and even hearing can be considered 
as hierarchies of maps (see Chapter 3). They 
are topographically related either to sensory 
surfaces like the retina, and/or to output 
arrays, like the motor homunculus of the cor¬ 
tex (see Chapter 5). Spatial mapping allows 
neurons representing different stimuli or 
output features at different levels of abstrac¬ 
tion to ensure that they relate closely to each 
other if they refer to the same point in space 
or to the same part of the body. Such spa¬ 
tial mapping is complemented by temporal 
rhythms, which add the ability to coordinate 
corresponding regions of each map to a flow 
of events taking place over precisely specified 
moments in time. Together, topographical 
maps and oscillatory coding make possible 
many kinds of spatiotemporal codes in the 
brain. Source : Adapted from Friston, 2003. 
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1.7 Two-way connections 

Most highways in the brain are two-directional. We 
tend to think of the visual system as a one-way flow 
of input, beginning at the retina, then flowing to the 
LGN, and then the primary visual cortex (VI) and so 
on. In fact, about 90% of the LGN-V1 fibers 'are run¬ 
ning the wrong way'. Above the LGN everything is a 
two-way highway. This is a dominant feature of the 
brain, and it is a great challenge to understand how 
two-way connections work. 

When the same topographical point is activated in 
two or more visuotopical maps, we are likely to get 
reentrant signaling , just as if many people with their 
own maps of a city were telephoning the same set of 
map coordinates to each other. Reentrant signaling 
sometimes is called 'resonant activity', and it is broadly 
consistent with the ubiquity of brain rhythms linking 
topographical maps. Regular rhythms and topographi¬ 
cal mapping may be two sides of the same coin, a bio¬ 
logical operating style that allows signals to be passed 
between similar points on many layers of maps. The 
body senses and motor output seem to work in much 
the same way, using topographical maps of the body 
and its surrounding space (see Chapter 5). 

Reentrant loops between similar maps can strengthen 
or inhibit each other. If all the map readers in our meta¬ 
phor agree on the coordinates of a location, your con¬ 
fidence in their reports will increase. Each confirmed 
message strengthens the combined signal and reduces 
errors and noise. But when map readers disagree on 
the coordinates, their information will tend to weaken 
each other. Active neuronal maps can engage in inhibi¬ 
tory interactions as well as excitatory ones. About 90% 
of the neurons in cortex are excitatory, but they could 
not function without numerous inhibitory neurons to 
regulate their activity levels, impose rhythmic pacing 
on excitatory activity, and sculpt inhibitory boundaries 
between excitatory islands of activity. 

The theory of Neural Darwinism (ND) is consistent 
with this perspective, with the added point that in ND, 
topographical maps evolve over time toward a better and 
better fit of the sensory input or motor output (Edelman, 
1989). In ND, neurons and synapses are strengthened if 
they make successful connections and weakened when 
they fail to do so. The same point applies to excitatory 
and inhibitory connections among competing popula¬ 
tions of neurons. Like biological evolution, ND allows 
for 'fitter' maps to win out over 'less fit' ones. 

As mentioned, brain maps plus rhythmic signal¬ 
ing constitute a spatiotemporal code. As we have seen, a 


television broadcast makes use of a spatiotemporal code. 
If you send a photo of yourself by cell phone, you are 
also using a spatiotemporal code. The brain must deal 
with the same engineering challenges that humans have 
learned to deal with using computers and telephones. 

1.8 How neurons synchronize 

Synchrony is a pervasive feature of the brain, appar¬ 
ently to coordinate patches of neurons at different loca¬ 
tions. Indeed, synchrony appears even when neurons 
are cultured in a laboratory dish, or when a thin slice of 
the thalamocortical core is kept viable by growth factors. 
Synchronous activity appears to be a self-organizing fea¬ 
ture of neurons in their natural environment. However, 
in the brain, synchrony has multiple signaling roles, and 
it appears to be under homeostatic control, like all other 
physiological parameters. The detailed workings of 
brain rhythms are still being studied. 

When two brain regions fire in synchrony with a lag 
time, the term phase locking is more accurate than syn¬ 
chrony. Sound waves echoing in a canyon are phase 
locked but not synchronous, because they echo back and 
forth with a brief lag time. Because neurons also take 
time to send their axonal spikes, there is a lag time in 
echoing among related brain regions, leading to phase 
locking rather than synchrony. Both synchrony and 
phase locking are commonly observed in the brain. 

Excitatory neurons can synchronize their firing 
when they are driven by a common inhibitory cell. 
Synchronized oscillations are believed to have many 
roles in the brain, in perception, attention, and memory 
(Figures 8.12 and 8.13). Synchronized groups of neu¬ 
rons firing at gamma rates tend to recruit additional 
neurons firing at the same rate (Llinas et ah, 2005)Faster 
rhythms are favored in this respect, because they allow 
for better temporal precision; beta and gamma provide 
neural 'clocks' that can detect small time differences 
compared to slower rhythms. 

Synchronized group oscillations reflect the way indi¬ 
vidual neurons operate. Single neurons can accumu¬ 
late inputs from other cells over about 10 milliseconds 
before they fire an action potential. If two neurons 
send their signals to a target neuron spaced more 
than 10 milliseconds apart, their effects simply fade 
before adding up to a new spike in the target neuron 
(see Chapter 3). This basic fact is believed to place an 
upper limit on the rate of neuron-to-neuron signaling. 
Relatively faster rhythms of the waking state, like 
theta, alpha, beta, and gamma, are in the right fre¬ 
quency range to recruit additional neurons. 



1.0 INTRODUCTION 


253 


(a) 

Gamma synchronization 

AAAAAAAA 



— 1 


— 1 — 1 — I — 1 —1—1—i— I- 

1 /N- 


1 /V 

Alpha synchronization 


; i 

1 /s. 


i 


(d) 


' ' ' 4 

AAAAA * 


AAAAA A 


(b) 



Attended 


1 


- 1 , * ,i » 

■ ■ ■ . ■ , 

[ i I_LJ_L 


1 

| M 11111111111 

VA'VWAWW'A 

I I IN II I jl I I 11 MM 

i iii11i!iiiii ! I 

—t- j j ‘ ’ H j ? r ! It j 

iii i mu Jiiii.ilji 


(b) 


"A lllllllllllllllllllllllllll II 

W 

iii mu mj iii. i 

1 1 1 

Iff 

1 

l l f 

1 111 1 j i.i 1 1 111 \ 

r T 

1 1 111 i 111 1 i 

j |i k 

W 

ii i i ii i i 

\ \ \ 




■ 1 ■ l I -- ■ , ■ -i- L-L 

—l-LJ-1—1-|—1- 


■ ■ ■ ■ ■ ■■ 


FIGURE 8.12 Synchrony in perception, attention, and mem¬ 
ory. (a) Shows how several neurons firing at the same time can 
add up to a gamma wave in the electrical field potential. Gamma 
synchrony allows for finer temporal resolution than alpha, 
(b) Shows how attentional mechanisms might work by selectively 
enhancing populations of neurons that drive target neurons at 
gamma rates. If you are looking at the red face on a sheet of white 
paper, neurons that are sensitive to the difference between red 
and white, those that fire to high visual contrast, and those that 
pick up faces will tend to fire in synchrony to enable the brain to 
encode the red face. If you then decide to pay attention to the red 
face, your brain can add to the activity of those sensory populations 
by firing additional cells in synchrony with the red face-sensitive 
populations. The blue face will not receive these additional pulses 
of activity timed with its feature neurons and will therefore not 
be enhanced, (c) Shows how synchronized neurons may retain 
a dynamic memory of a visual stimulus for some time (seconds 
or minutes). The sample face triggers populations of feature- 
sensitive neurons that fire in synchrony with each other. When the 
sample face is removed, some of those feature neurons continue 
to fire for some seconds or minutes. When the recognition probe 
is presented, the relevant neurons are already primed to respond 
in synchrony to the recognized stimulus. There is direct brain 
evidence for such short-term memory mechanisms. Source: Jensen 
et al, 2007. 


FIGURE 8.13 Inhibitory cells synchronize excitatory neu¬ 
rons. How inhibitory neurons drive synchrony in excitatory cells, 
(a) Coding by synchrony. If two excitatory cells (black) display ran¬ 
dom spike trains, they will rarely drive a common downstream cell 
above threshold (gray). By contrast, when synchronized by an inhib¬ 
itory interneuron (red) these neurons, firing at the same overall rate, 
will now reliably activate the downstream neuron, (b) Phase coding. 
If five pyramidal neurons (black) are firing at different rates, the 
introduction of a rhythmic inhibitory cell will produce a rate-to-phase 
transform for the first spike in each cycle. The angled red line indi¬ 
cates the range of phase encoding made possible by this mecha¬ 
nism. Source: Mann and Paulson, 2007. 


Regular population rhythms, like the ones we 
observe in the EEG, are believed to require both exci¬ 
tatory and inhibitory neurons. The pyramidal (excita¬ 
tory) neurons of the cortex are surrounded by many 
inhibitory interneurons, which are needed to shape 
regular rhythmic waveforms. Figure 8.12a shows how 
neuronal firing adds up to gamma synchrony in the 
EEG or the local electrical field. Alpha synchrony is 
slower and therefore has less temporal precision. 

In principle, attentional control regions in the brain 
can synchronize their output with target areas; for 
example, the attentional network discovered by Posner 
and coworkers may dance to the same beat as face- 
selective visual maps of the temporal lobe (Figure 8.31). 
Figure 8.12b shows that adding to the overall synchrony 
strengthens the red face signal and decreases the blue 
face activity by breaking up its synchronous input activ¬ 
ity. Finally, in Figure 8.12c, a synchronous population 
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can respond to a stimulus and may keep running for 
some seconds or minutes after the stimulus has ended. 
Synchronous wave activity can therefore store a tem¬ 
porary memory, but it may fade fairly quickly and may 
also be vulnerable to interference from other stimuli. 

These are only some of the coding possibilities of 
synchronous neurons and populations. Obviously, 
these are hypotheses that require supportive evidence, 
as we will see. 

1.9 Synchrony for gain control 

Gamma synchrony may amplify neuronal population 
amplitude because synchronized neurons can add 
to each other's activity. Synchrony tends to increase 
the size of the signal in much the way a microphone 
that picks up sounds from a loudspeaker will tend to 
amplify the signal over and over again. In biological 
systems, such self-amplifying feedback can never be 
allowed to run out of control for the same reason that 
sound systems can't allow infinite self-amplification. 
Audio systems have control circuits to prevent ampli¬ 
fier feedback from overloading the speakers, not to 
mention the ears of the listeners. Epileptic seizures may 
actually represent the wrong kind of self-amplification 
of slow rhythms, interfering with normal brain func¬ 
tions and even leading to a loss of consciousness. 

We should not leave the impression that the wak¬ 
ing state is completely synchronized at high frequencies. 
Rather, synchrony appears in local task-related patches of 


the brain, surrounded by nonsynchronized populations 
of neurons. If a task like vision needs to coordinate multi¬ 
ple visual maps, gamma synchrony may appear in mul¬ 
tiple locations in the visual cortex (see Figure 8.23). Or as 
we will see later, if a conscious brain surgery patient has 
been asked to name a visual object, we may see transient 
gamma synchrony between areas for object recognition 
and speech production. 

The raw EEG shows only the surface waves of a 
deep and turbulent ocean. Underneath the visible EEG 
there are multiple oscillatory streams interacting over 
a wide range of frequencies (0.01-1000 Hz have been 
observed). Some of these under-the-surface waves are 
locked in synchrony with each other, some of them are 
phase-locked, many are transiently coordinated, others 
show cross-frequency coupling. The metaphor of a tur¬ 
bulent ocean is a useful first approximation, but unlike 
the ocean in a biological brain, there are many functional 
activities going on all the time. Knowledge about the 
roles of brain rhythms is building at remarkable speed. 

In Figure 8.14 the entire visual stimulus (a) can actu¬ 
ally be seen as a candlestick or (b) as a single face look¬ 
ing toward us (c) or as two faces looking at each other. 
Thus, the identical physical input has three interpreta¬ 
tions. Each perceived object involves a different figure- 
ground grouping. If object-sensitive neurons pick up 
the candlestick (Object 1), they can drive other neurons 
to synchronize with it and inhibit the other objects. 
The coalition of synchronized maps that fires the larg¬ 
est signal may either out-vote the nonselected objects. 
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FIGURE 8.14 How synchronized 
neurons may bind sensory features. Brain 
rhythms may "bind' (unify) multiple features 
of a visual stimulus into one coherent visual 
object. We know that the visual brain takes 
apart a physical stimulus into component 
features: location, color, pixel-level details, 
edge contrasts, and orientation, and ulti¬ 
mately, object identity (see Chapter 6). "Object 
identity', like seeing a coffee cup as a whole, 
combines all those features into a single 
'gestalt' entity For that reason, object repre¬ 
sentation has been thought to 'bind' multiple 
visual features. This is sometimes called 'the 
binding problem'. It is associated with the 
question of visual consciousness, because 
we tend to be conscious of whole objects and 
events rather than isolated features floating 
in space. Source : Engel et al, 2001. 
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or it may actively inhibit them (Engel et ah, 2001). We 
will see direct brain recording evidence for synchro¬ 
nized firing of visual feature sensitive neurons below 
(see Figures 8.15 and 8.16). 

Some authors suggest that all brain oscillations - 
waking, dreaming, and sleeping - are carried or mul¬ 
tiplexed by very slow oscillations, cycling at less than 
0.5 Hz (Steriade, 2000). As we will see, there is direct 
evidence for multiplexing of gamma waves on theta 
and perhaps gamma on alpha as well. Since the gamma 
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FIGURE 8.15 Multiplexing faster waves on slower ones. 
Fast waves may 'multiplex' on top of slower waves, like the ampli¬ 
tude modulation of radio signals. This is known by several other 
names. It appears to be a common coding strategy in the brain. 
Source : VanRullen and Koch, 2003. 
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range is expanding to 120Hz, 200 Hz, or even 1 000 Hz, 
it seems likely that we will see many more kinds of mul¬ 
tiplexing in the brain as recording techniques improve. 

The basic advantage of multiplexing is to lower the 
firing threshold of downstream neurons by reducing 
their membrane polarity (see Chapter 3). Thus, theta 
waves may spread to some target population and make 
them slightly more sensitive, after which added gamma 
waves may push them over the firing threshold. If the 
target neurons are able to follow the input at gamma 
rates, they may then produce their own synchronized 
gamma burst, which may in turn 'ignite' a larger pop¬ 
ulation of neurons. For example, VanRullen and Koch 
(2003) have shown that visual signals can be detected 
better if they occur at the peak of the endogenous alpha 
wave. One plausible explanation is that many regions 
of the visual brain oscillate at alpha rates during resting 
states, and that the baseline voltages of large numbers 
of neurons are gently being raised and lowered at 12 
to 14 Hz. Certainly that is the view of slow oscillations 
propounded by Steriade (2000) and others. 

The cortex appears to have a number of microcir¬ 
cuits that can perform more complex computations 
like coincidence detection and even elementary logic 
and arithmetic operations like the ones that are built 
into basic computer chips (Gabbiani et ah , 2002). The 
present discussion is focused on only basic oscillatory 
aspects of neural computation. 

We will now look at some details of how the thal¬ 
amocortical core of the brain works in the three major 
states. 
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FIGURE 8.16 Waking: Stimulus-specific reverberation during 
early learning. A dynamic neural circuit for the first phase of learn¬ 
ing. During the waking state, working memory is believed to be sus¬ 
tained by a recurrent circuit, including cortex, thalamus, and basal 
ganglia (striatum), enhanced by neuromodulation originating in the 
substantia nigra (SNr) and globus pallidus (GPi). While visual input 
enters the cortex in the occipital lobe, integration of visual objects is 
believed to occur in the inferior temporal lobe (called area TE in the 
macaque brain, shown here). Parietal involvement is required for 
spatial localization of visual events in body space and for manual 
reaching and manipulation. Prefrontal involvement is required for 
overall integration, voluntary attention, and motor control (execu¬ 
tive functions). Notice that glutamate and GABA are the two major 
neurotransmitters shown. Stabilization of this circuit may require 
NMDA receptors for increasing the efficiency of active synapses. 
The hippocampal complex (not shown) is required for episodic 
learning and, over time, for semantic memory Memory consolida¬ 
tion during SWS is needed to make enduring memories by protein 
synthesis which makes new synapses and other cellular structures 
needed to sustain long-term memory Source : Wang, 2001. 


The contents of our waking consciousness include 
the immediate perceptual world; inner speech and vis¬ 
ual imagery; the fleeting present and its fading traces in 
immediate memory; bodily feelings like pleasure, pain, 
and excitement; autobiographical memories when 
they are remembered; clear and immediate intentions, 
expectations, and actions; explicit beliefs about oneself 
and the world; and concepts that are abstract but focal. 
Obviously, we are conscious of much more than we 
report at any given time, but humans can answer ques¬ 
tions about a remarkable range of conscious events - 
the sight of a single star on a dark night, the differ¬ 
ence between the sounds 'ba' and 'pa', and the goal of 
studying for an exam. All sensory systems do a great 
deal of unconscious processing, leading to conscious, 
reportable sights, sounds, and feelings. 
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Recalling an autobiographical memory also results 
in conscious experiences, like your memory of seeing 
this book for the first time. Inner speech and imagery 
have conscious aspects. Asking someone to rehearse a 
list of words usually results in conscious inner speech. 
People vary in the vividness of their visual images, 
but dreams, for example, are classically reported to 
have vivid imagery (Stickgold et al., 2001). Finally, 
action planning can have conscious components, espe¬ 
cially when we have to make choices about our actions 
(Lau et al., 2004a, 2004b). Normally, we may not have 
to think much about the action of walking, but if 
we break a leg, even standing up and walking can 
become a consciously planned process (Sacks, 1984). 

There is good evidence that we have a spontaneous 
inner flow of conscious contents that constantly compete 
against the perceptual world (Singer, 1994; Mazoyer et al., 
2001; Fox et al., 2005). In sum, in everyday life, there is 
ongoing competition between inner and outer events, 
between perceiving and responding, and between 
simultaneous input streams. Attentional selection seems 
to operate every moment of the waking day. 


2.1 Realistic thinking 

While dreams are notoriously unrealistic, waking con¬ 
sciousness has long been considered to be the realm 
of logical, mature, reality-oriented thought. There is 
considerable truth to that, even though we also com¬ 
monly engage in waking fantasies, daydreaming, 
emotionally driven thinking, and mind-wandering. It 
is notoriously difficult to focus on just one task with 
full concentration, and there may be good biological 
reasons for that. Humans did not evolve in a world 
of classrooms and book learning. Paleolithic hunter- 
gatherers had to be constantly alert for unexpected 
dangers and opportunities; focusing completely on one 
thing to the exclusion of all others would have been 
dangerous. Humans have a tendency to lightly scan 
the sensory world, rather than doggedly following 
a single goal or a single train of thought. At the same 
time, our large brains give us enormous adaptability. 
We can learn to control our attention, at least for some 
period of time, and traditional cultures train children 
in cognitively demanding tasks like hunting, path-find¬ 
ing, identifying hundreds of plants, traveling long dis¬ 
tances, making tools, preparing food, and adhering to 
numerous social customs. Those habits require learn¬ 
ing and executive attention. They also place a premium 
on understanding the world realistically. However, the 
brain basis of 'realism' has not been studied sufficiently 


so far. As a result we do not understand how we 
can watch a horror movie without being frightened 
(because we know it is an illusion) and compare that 
to realistic experiences. Given the tools we have today, 
that question can be addressed empirically. 

In a precarious world it requires self-control not to 
allow feelings like fear, hunger, loneliness, or anger to 
take over. Children in hunter-gatherer cultures often 
are trained to endure privation and emotional stress. 
Our brains evolved to cope with such conditions. It is 
only very recently that human beings have lived more 
secure lives, with reliable sources of food, social sup¬ 
port, and protection from danger and discomfort. 

The waking brain has been described as being 
'poised on the cusp of a transition between ordered 
and random behavior' (Table 8.1). This delicate state 
'optimizes information transfer and storage ... (includ¬ 
ing) both prolonged periods of phase-locking and 
occurrence of large rapid changes in the state of global 
synchronization...'. (Kitzbichler et al., 2009). The wak¬ 
ing thalamocortical state is optimized to combine such 
capacities. Rhythmic synchrony and phase-locking 
allow the brain to relate different time scales to each 
other. Thus, the piccolo of the brain orchestra can play 
in rhythm with the bass drum. 

The last decade has seen an avalanche of findings 
on brain rhythms in the sensory and motor systems in 
the hippocampal memory system, selective attention, 
and executive control. These findings come from direct 
brain recordings in animals and humans, from EEG 
and MEG, and from combining these electromagnetic 


TABLE 8.1 Major features of the conscious 
waking state 


1 Very wide range of conscious contents 

2 Widespread brain effects beyond the specific contents of 
consciousness 

3 Informative conscious contents 

4 Rapid adaptivity and fleetingness 

5 Internal consistency 

6 Limited capacity and seriality 

7 Sensory binding 

8 Self-attribution 

9 Accurate reportability 

10 Subjectivity 

11 Focus-fringe structure 

12 Consciousness facilitates learning 

13 Stability of conscious contents 

14 Object and event-centered 

15 Consciousness is useful for voluntary decision making 

16 Involvement of the thalamocortical core 


Source : Seth and Baars, 2005. 
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recording methods with the spatial advantages of fMRI. 
Interlocking rhythms have been observed in single neu¬ 
rons, in neuronal circuits and arrays, and even in entire 
brain lobes. Rhythmic activities in the core brain are the 
most obvious markers of different states of conscious¬ 
ness, and recent evidence suggests that attentional selec¬ 
tion operates by way of synchronized brain rhythms as 
well. The slow oscillatory state of deep sleep seems ded¬ 
icated to consolidating memories of the conscious expe¬ 
riences of the preceding day. As we will see, even sleep 
seems to utilize fast rhythms during the peak of slow 
delta wave to consolidate the conscious experiences of 
the preceding day. 

Table 8.2 lays out various kinds of consciously 
mediated cognition - pretty much all processes we 
study. Notice the unconscious aspects of these partly 
conscious tasks. Both conscious and unconscious com¬ 
ponents are likely to be crucial for normal, flexible, 
adaptable cognition. However, subliminal perception 
and learning, for example, may take place for highly 
significant and stereotyped stimuli. 

2.2 Orientation to place, time, and persons 

Our basic knowledge about conscious experiences comes 
from what people can tell us (see Chapter 2). When a 
patient with possible brain damage is clinically examined, 
a neuropsychologist will begin by administering a 'mini¬ 
mental status exam'. It spells out what we expect from a 
conscious person with a healthy brain (McDougall, 1990): 


1 Orientation to time, place, and person: 'What day 
of the week, date, and season is it'? 'Where are you 
now'? 'Who are you'? 'Who am I'? [the examiner]. 

2 Simple working memory: 'Remember these three 
objects ...'. 

3 Attention and calculation: 'Count down by sevens 
starting from 100' (100, 93, 86, 79, 72 ...). 

4 Intermediate recall: 'What were the three objects I 
mentioned before'? 

5 Language: Ask the patient to name a wristwatch 
and a pencil, to repeat a sentence, to follow three 
simple commands, to read and understand a sim¬ 
ple sentence, and to write their own sentence. 

6 A basic visuomotor skill: Copying a drawing of a 
line pentagon. 

Notice that normal voluntary control is implied by all 
of these tasks. People with frontal lobe deficits may 
have difficulty doing them because of impaired execu¬ 
tive functions. As basic as the questions may seem, they 
cover sensory perception, voluntary control, working 
memory, immediate and intermediate memory, long¬ 
term retrieval, mental effort, language, attention to daily 
events, regular updating of memory as time goes on, 
mental imagery, spatial skills, and more. All these tasks 
require both the waking state (alertness) and consciously 
mediated cognition. They are impaired by drowsiness, 
attentional lapses, SWS and REM dreaming, as well as 
by drugs and brain disorders that impair consciousness. 

Normal waking is the necessary condition for all 
the cognitive processes we discuss in this book. 


TABLE 8.2 Consciously mediated cognition 


Task 


Conscious aspects 


Unconscious aspects 


Working memory 

Spontaneous problem-solving 

Sensory feature binding, posterior cortex 
Sensory-spatial integration, parietal lobe 
Sensorimotor skills (e.g. speaking) 

Comparing conscious experiences that 
occur at different times 
Voluntary attention 


Automatic attention 
Implicit learning 


Sensory input, response planning and output, 
momentarily mentally rehearsed item 
Problem definition, realizing the solution 
(the Aha'! experience) 

Results of binding 
Same 

Acquisition phase and making changes in 
routine tasks 

Novel memories, and novel relationships 

Effortful control of attention, or maintaining 
attention against contrary tendencies, such 
as fatigue, drowsiness, or more attractive 
topics of attention 

The results of attention are commonly 
conscious (but not always) 

Conscious cues to the implicit regularities or 
inferences 


Preperceptual, unrehearsed, basal ganglia 

Problem incubation 

Detailed process of integration 
Same 

Predictable components after extensive practice 

Well-established memories and procedures 

Routine, automatic, noncompeting, and highly 
predictable control of attention 

Detailed control of automatic attention 
Automatic or implicit information to be learned 
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2.3 Waking is for learning; sleep enables 
memory consolidation 

Although waking is by far the best time for learning, 
sleep and REM dreams enhance memory consolida¬ 
tion. Memories are believed to be unstable and vulner¬ 
able to interference in the early hours after learning. 
Early memory is thought to depend upon active, reso¬ 
nating neural circuits (Figure 8.16). However, some 
researchers believe that waking may also allow some 
memory consolidation. After about a day, learned 
material appears to be 'consolidated' - made more 
enduring - a process that is thought to require pro¬ 
tein synthesis. Lasting memories involve structural 
changes to synapses and cells and sometimes to entire 
brain networks. Sleep appears to enhance memory 
consolidation by protein synthesis, which requires 
gene expression in numerous brain cells. 

Although all states of consciousness have multi¬ 
ple functions, the conscious waking state is by far 
the most adaptable one. As we will see in Chapter 9, 
every region of the brain has its own ability to adapt 
and learn. Therefore, there are many kinds of learn¬ 
ing. But to explore new kinds of information and com¬ 
mit them to memory, we are constantly making things 
conscious. In everyday life, that's what we mean by 
'paying attention'. Memory researchers point out that 
most of our learning is 'incidental'; that is, it occurs as 
a benefit of paying attention (Kristjansson, 2006). We 
are learning during most of our waking moments, as 
we can show by using sensitive measures of episodic 
(experiential) memory. But we are certainly not delib¬ 
erately memorizing things in every waking moment. 
Memorizing is only one way to make learning happen. 
Most of the time it seems that we learn things just 
by paying attention to them - which means becom¬ 
ing conscious of them. Memory researchers since 
Hermann Ebbinghaus have pointed to the central 
role that conscious attention plays in normal learn¬ 
ing. Although there is some evidence for unconscious 
acquisition of information, we will focus on the pow¬ 
erful role of conscious attention in mobilizing hip¬ 
pocampal and cortical mechanisms of learning (see 
Chapter 9). 

2.4 Attention and consciousness commonly 
enable learning 

How is it that you can learn the material in this book? 
Human beings really do not know. What we do in 
daily life is simply pay attention to new and interesting 
things - exploring them with our senses, rehearsing 


them mentally, and repeatedly directing our atten¬ 
tion to them - and magically, learning seems to 
occur. The next day, we suddenly realize that yes¬ 
terday's new information seems more familiar. We 
must have learned it. Orienting tasks are important 
to enable learning. 'Mere exposure' to information is 
often enough to enable recognition memory. In cogni¬ 
tive science jargon, most of our everyday learning is 
incidental. 

What we generally do is just pay attention to new 
material, even if it seems hard to understand. The big¬ 
gest challenge is to pay attention to new and difficult 
information and to be patient enough to allow our 
brains to wonder, ask questions, and, ultimately, com¬ 
prehend any new material. Once we perceive and com¬ 
prehend something new, learning seems to occur. Brain 
evidence indicates that spontaneously paying attention 
to some sensory content also activates the hippocampal 
complex (Stark and Okado, 2003). 

We can direct our attention voluntarily, although 
it sometimes takes a noticeable effort to do so (see 
Chapter 12). Selective attention generally leads to 
conscious experiences (earlier), and it seems to be the 
act of making things conscious that enables learning. 
Indeed, 'episodic and declarative learning' are defined 
as 'learning of conscious events', and we know that the 
hippocampus seems especially adapted for that (Squire 
et al., 2008; see Chapter 9). Though there is reliable evi¬ 
dence for subliminal perception and learning, we will 
limit this discussion to episodic and declarative mem¬ 
ory of conscious events. 

Cognitive neuroscientists believe that declarative 
and episodic learning generally occurs something like 
this (Seitz and Watanabe, 2005): We pay attention to 
new information so that it tends to become conscious. 
As soon as we experience the new information with 
enough clarity, our brains are able to store it. Repeated 
attention to new or difficult material often is needed 
before we get a sense of clarity. By using sensitive meas¬ 
ures of episodic and declarative (conscious) memory, 
like recognition measures, we can show that humans 
learn very quickly after clear conscious experiences of 
new information. There is reliable evidence for sublimi¬ 
nal learning, but here we will focus on learning of con¬ 
scious events, or of material that is cued by conscious 
events. 

That does not mean we can recall all memories on 
cue. It does mean that we can recognize consciously 
experienced events far above chance. Recognition 
tests are easy because they simply reinstate the same 
experience we had before. We don't have to voluntar¬ 
ily search our memories at all. All we have to do is say. 
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'Yes, I've seen it before', or 'No, I haven't'. Simple con¬ 
scious experience of the material to be recognized will 
trigger automatic memory search. The yes/no deci¬ 
sion is made by means of familiarity. 

For example, you may be surprised to recognize a 
scene in a movie you may have seen 10 years ago. It 
wasn't necessary for you to memorize the movie scene. 
All you had to do was to watch it once - consciously - 
and then see it again 10 years later. Consciousness of 
an event seems to be enough to establish it in memory. 
Much the same is true of recognizing yearbook pho¬ 
tos of high school classmates, news photos, headlines 
from years ago, and the like. 

Unfortunately for students, academic exams rarely 
use recognition tests. Rather, exams test associative 
recall by asking questions like, 'What is the capital of 
France'? College exams would be a lot easier if they 
gave us a part of the answer, like the partial recogni¬ 
tion item: 'Par... is the capital of France'. 

Associative recall tests give much lower estimates 
for accurate memories than recognition tests. That is 
why academic exams are difficult: It is therefore not 
our stored memories that are at fault but our ability to 
retrieve them on demand. By analogy, you can file a 
book randomly in a giant library. The book (the mem¬ 
ory) is somewhere in there, but you cannot retrieve it 
on demand unless you have a very good retrieval sys¬ 
tem, like a search engine or a card catalog. Otherwise, 
the book might as well be lost forever. 

Notice that simply paying attention does not 
always make things conscious. For example, we can 
pay attention to an ambiguous figure without seeing 
one of its interpretations (see Chapter 1). As a result, 
we may never learn the nonconscious interpretation of 
the figure as a conscious, episodic memory. Learning 
a complicated subject like brain science is often like 
learning an ambiguous stimulus and never quite 
seeing the unaccessed interpretation. At first, new 
material may seem vague, hard to understand, or con¬ 
fusing. Then we may spend some time thinking about 
it, paying attention to it, trying to draw it by hand, 
or answering questions about it. Over time, a clearer 
sense of meaning tends to come. At the point when we 
become clearly conscious of the desired information, 
learning tends to follow quickly. Most of our effort in 
studying new material is therefore devoted to the task 
of comprehension. Once we understand things clearly, 
memory tends to come along. 

In sum, paying attention may be a means toward 
conscious clarity, which in turn enables episodic and 
declarative learning. The hippocampal complex is 
believed to turn conscious experiences into memories. 


At first, hippocampal memory is believed to be unsta¬ 
ble, or labile. Consolidation serves to make long-lasting 
changes in the hippocampus and to transfer learning 
to the neocortex. Theta oscillations are believed to play 
a major role in both local hippocampal functions and 
in long-range coordination between the hippocampus 
and neocortex. 


2.5 Losing consciousness of predictable 
routines 

Although paying attention to a stimulus often leads 
to different ways of experiencing it, there is a funda¬ 
mental case where the opposite happens: The case of 
practiced habits. As a rule, the more predictable a sen¬ 
sorimotor skill becomes, the less of it will be conscious. 
That is a common-sense observation about learning to 
ride a bicycle and even learning to speak a new lan¬ 
guage. The fading of conscious access to habitual skills 
is commonly called 'automaticity' and it goes along 
with a loss of precise voluntary control over habitual 
details (see Chapter 2, Section 5.1). 

The world around us becomes more predictable as 
we grow up and learn to understand it. If everything 
were completely known and predictable we might 
not have to pay attention at all. Our brains could just 
go on automatic. Fortunately the real world is full of 
changes and challenges, and we find ourselves con¬ 
stantly choosing the most important and novel events 
to pay attention to. The stream of consciousness is all 
about our personal news headlines. We tend to dwell 
on whatever is novel or significant to us. 

Repetitive events tend to fade from conscious¬ 
ness unless they have special significance. The itch 
of a mosquito bite may not fade quickly, because the 
saliva of the female mosquito triggers the pain sys¬ 
tem. Pain is inherently significant, like hunger or the 
taste of chocolate. We do not habituate rapidly to 
biologically significant events, but neutral ones 
can fade quite rapidly, sometimes in seconds. Even 
though gravity constantly triggers our vestibular 
(balance) sense, we rarely pay attention to it because 
it has become predictable. The body in relation to 
gravity becomes a focus of attention again for astro¬ 
nauts, parachute jumpers, and children riding a Ferris 
wheel. 

Schneider (2009) writes that 

The concepts of automaticity, controlled processing 
and consciousness are tightly coupled .... Automatic 
processing ... supports parallel, low effort process¬ 
ing that is difficult to control. Controlled processing 
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operates from a set of 'domain general' areas that 
exhibit serial effortful processing in novel or changing 
situations. 

(However), automatic processes in the form of 
automatic attention can activate conscious awareness 
to stimuli that have been unattended. 

The last point is very important. We lose conscious 
access to automatic routines, but some of them are 
attentional routines - that is, they function to bring 
things to consciousness. Automatic eye movements 
are a good example. Many eye movements are auto¬ 
matic, and as they bring a moving truck into our field 
of vision, the sight of the truck becomes conscious. 

In those cases, of course, automaticity actually ena¬ 
bles important information to become conscious. But 
more generally, practice leads to automaticity and a 
drop in conscious access to the highly practiced skill. 
Much of our ordinary mental life recruits automatic 
routines that are invisible to us, but which enable all 
of our daily activities. 

Brain studies of practiced skills show that cortical 
fMRI declines drastically when the skill fades from con¬ 
sciousness. Figure 8.17 shows a single subject's brain 
scan before and after learning a predictable auditory 
search task to the point of automaticity. As a result of 
practice and learning, the fMRI (BOLD) signal of an 
entire network of cortical regions is observed to drop 


Controlled processing Automatic processing 



FIGURE 8.17 fMRI activation before and after practice. Con¬ 
trolled and automatic versions of the identical task can be compared 
in the same subject. When the auditory search task is novel it allows 
more detailed voluntary control, more conscious access, and more 
widespread cortical activation. Cole & Schneider (2008) have shown 
that voluntary control processing involves a network of domain- 
general areas of cortex, shown above. They naturally divide into 
frontal lobe activations (in front of the central sulcus) and BOLD 
activity in the posterior part of the parietal lobe. The same frontal 
and parietal regions are active in conscious, compared to uncon¬ 
scious, stimulus processing. These areas are called "domain general' 
because they function in different tasks, using visual or auditory 
stimuli, with either simple or complex stimuli and tasks. Source : 
Schneider, 2009. 


below detection threshold. Such marked drops in 
cortical activity commonly are found in highly prac¬ 
ticed, predictable tasks. 

Notice, however, that a part of the auditory cortex 
remains active even after the scanning task becomes 
automatic, namely the upper rear ridge of the pos¬ 
terior temporal lobe. The reason is that the auditory 
input continues to be analyzed. However, the local 
cortical activation does not seem to spread widely, as 
fully conscious events seem to do (see Figure 8.17). 

Figure 8.17 raises some important questions. One 
is about the voluntary control network, which we 
discuss in more detail in Chapters 11 and 12. As you 
might expect, a number of control-related regions 
of cortex cluster around the motor strip, just in front 
of the central sulcus. They include the dorsolateral 
prefrontal region (dl-PfC), a part of the insula (ante¬ 
rior insula), pre-supplementary motor cortex (PMC), 
dPMC (dorsal premotor cortex), and the inferior fron¬ 
tal junction (IFJ). Just over the top of the hill of the 
hemisphere, we see activity in premotor cortex (dor¬ 
sal part) and in anterior cingulate (AIC). It is easiest to 
think of all these regions as different levels of motor 
maps, some very specific to the muscles, and other at 
higher levels, including simply the 'urge to move'. DL- 
PFC is involved in all effortful voluntary tasks. Notice 
also that in the speaking hemisphere these regions 
border on Broca's area, the speech production region 
of cortex, which is positioned close to the motor cortex 
for the mouth and tongue. These frontal regions have 
complicated names, but they follow a simple rule 
because they are all frontal, and intimately involved 
with action control. 

Notice that these frontal areas use both faces of each 
hemisphere, the lateral (outside) and medial (inside) 
surfaces. Essentially they cover both hillsides of 
the same hill. Because the cortex is basically one large 
surface, its hills and valleys are not as important as the 
continuous surface itself. 

Finally, we see BOLD activity to the rear of these 
frontal areas, in PPC, the posterior parietal cortex. You 
can think of this region as the action planning center 
before the human frontal lobe bulged out as far as it 
has, since earlier primates. Since the brain contains 
its own history, we can think of the human brain as 
having two action control centers, both of them feed¬ 
ing into the motor strip, which has the most direction 
connections to motor nerves traveling down the spi¬ 
nal cord and the cranial motor pathways to control 
the face, head, and neck. Because the vast forward 
expansion of the frontal lobes is quite recent (on an 
evolutionary time scale) we use both posterior parietal 
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and prefrontal control centers. The posterior parietal 
cortex is involved in planning, mental imagery, and 
sensorimotor integration. All those functions partici¬ 
pate in controlled (voluntary) processes, and become 
less prominent when a practiced, predictable task 
becomes less conscious (Coulthard et al., 2008). 

In the automatic task on the right side of Figure 8.17, 
has the brain stopped processing the auditory search 
task at all? Clearly not. While cortex is much less 
active, automatic routines are controlled by the basal 
ganglia, cerebellum, and spinal circuits. They may still 
use sensory input, as indicated by the active auditory 
cortex. But such highly practiced loops can apparently 
become almost entirely subcortical. It is just uncon¬ 
scious or much less conscious, with less detailed vol¬ 
untary control and less cortical activity. 


2*6 Implicit learning requires 
consciousness, too 

The term implicit learning can be misleading. It does 
not mean learning without consciousness. Rather, 
implicit learning involves conscious input, leading 
to implicit (unconscious) inferences that are learned, 
along with the conscious aspects of the task. 

When we practice a complex sensorimotor skill, as 
in sports, dancing, typing, reading, or musical perfor¬ 
mance, we do not become conscious of all the details 
of the task. Rather, we make only a part of the task 
conscious and allow automatic aspects to be recruited 
unconsciously. What scientists call 'explicit', 'episodic', 
or 'declarative' learning involves conscious learning - 
but there is no exclusively conscious learning. Conscious 
and unconscious processes always go together. What 
is technically called 'implicit (unconscious) learn¬ 
ing' often comes as a side effect of conscious input. 
Conscious events may generally guide the learning 
process, even for implicit learning. 

For example, the grammar of our first language 
is learned implicitly (unconsciously). Young chil¬ 
dren acquire language with remarkable speed but 
without having to know explicitly which words are 
nouns, verbs, or adverbs. Children do pay attention 
to new words and sentences and certainly become 
conscious of them when they speak and listen. Good 
caregivers constantly call kids' attention to new words 
and new experiences. Thus, consciousness is con¬ 
stantly involved in language learning, but the regu¬ 
larities of grammar seem to be learned implicitly , as 
a side-benefit of paying attention and making things 
conscious. 


2*7 Fast rhythms coordinate waking tasks 

The raw EEG of waking often is called 'beta' or 
'gamma' activity, but it is in fact a complex and vari¬ 
able mix of many periodic and nonperiodic signals. 
We can think of the raw EEG as a choppy sea. In sci¬ 
ence, the ability to analyze complex waveforms at 
high speed is a recent development, dependent on 
powerful computers using Fourier analysis and simi¬ 
lar techniques. As a result, current scientific articles 
rarely refer to the raw (unanalyzed) EEG any more. 
That can lead to real confusion, because we still tend 
to think of EEG in terms of pure waveforms. To follow 
the most recent practices, when we refer to 'beta' or 
'gamma', we really are talking about a band of frequen¬ 
cies, beta being about 13 to 25 and gamma, 26 to 120 
or so. There is no perfect agreement on the boundaries 
between standard frequency bands, and there prob¬ 
ably should be no rigid numbers until we have a better 
understanding. New studies keep expanding to new 
wave bands that play important roles in the brain. For 
example, recent work shows that very-high frequency 
bursts are important (up to 1000 Hz) and that very- 
low frequency waves, down to .01 Hz, are also present 
in both waking and sleep. Keep in mind that brain 
rhythms are a moving target, as new evidence appears 
with remarkable rapidity. 

Evolution constantly reutilizes biological mecha¬ 
nisms for new functions. Ancient parts of the brain, 
like the hippocampus - part of the reptilian small 
brain - already show both gamma and theta waves 
(4-7Hz). It is likely that the neocortex, the vastly 
expanded 'new cortex' of mammals, borrowed and 
made novel use of those ancient rhythms. The major 
brain rhythms are found very widely in the animal 
kingdom. They are highly conserved among species 
and therefore likely to be important. 

High levels of gamma oscillations have now been 
observed for sensory processing, attentional enhance¬ 
ment of sensory input, and both working and long-term 
memory. More recent work shows both high intensity 
of gamma activity and gamma synchrony, the simul¬ 
taneous up-and-down rhythm of two different brain 
regions at frequency rates above 20 to 30 Hz. Synchrony 
is both natural and useful for signaling purposes in an 
oscillatory system like the brain. Sometimes perfect 
synchrony is not attainable, so that there is a brief time 
lag between the peak of the wave in one place (like the 
hippocampus) and another place (like the frontal lobe). 
In those cases, the better term is phase locking or phase 
coherence , a little bit like a syncopated 'off-beat' rhythm 
in music. It is synchrony with a time lag. 
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Individual neurons have a temporal integration time 
of about 10 ms, the period when dendritic inputs can 
add up to increase the probability of a single axonal 
output spike (see Chapter 3). A group of intercon¬ 
nected neurons can strengthen each other's firing rates 
between 30 and 100 Hz by supplying synaptic inputs 
within the 10-ms window. If two excitatory neurons are 
signaling each other at a rate of 50 Hz, for example, it is 
possible to sustain an excitatory feedback loop, because 
converging signals can arrive within the critical 10-ms 
period. However, neuronal firing rates below 30 Hz may 
not be integrated by target neurons, because different 
spikes may arrive too late to have additive effects. It is 
therefore believed that a group of neurons firing in the 
beta-gamma range will exert a stronger drive to down¬ 
stream neurons than lower frequencies. Obviously, real 
brain networks are more complex and have inhibitory 
as well as excitatory elements. Nevertheless these basic 
points apply to neurons in general and have gained a 
good deal of direct empirical support. 

Radio transmission has some similarities to oscilla¬ 
tory synchrony in the brain. Traditionally, the power¬ 
ful electromagnetic waves used to broadcast radio use 
either frequency modulation (FM) or amplitude mod¬ 
ulation (AM). In both cases, a fundamental frequency 
(the number on your radio dial) acts as a carrier 
wave for faster changes in frequency or amplitude, 
which reflect the voice or music signal. In your radio 
receiver, an electronic circuit is tuned to the frequency 
of the broadcasting station, and it is able to filter out 
the added, modulating signal that rides on the carrier 
wave. You end up listening to the songs and never 
hear the carrier frequency. 

These similarities are not accidental. They reflect the 
basic physics of waves, which apply to brain rhythms 
as well as electromagnetic radiation. The existence 
of AM and FM radio suggests at least two ways in 
which brain rhythms may process information in the 
brain. But there are many more coding schemes. Brain 
rhythms could serve as clocks, and they can use single 
pulses or a series of pulses like Morse code. Different 
neurons may use signals in different ways, perhaps in 
combination with different molecules and synapses. 

Television is an example of a spatiotemporal code, in 
which the broadcast signal scans across every line of 
the screen from top to bottom. Computer screens use 
similar spatiotemporal coding. Brain rhythms are also 
likely to coordinate visuotopic maps, somatotopical 
maps, and motor maps. As we have mentioned, the 
brain is rich in topographical maps, which represent 
sensory input arrays or neuromuscular maps at vari¬ 
ous levels of abstraction (see Chapter 5). 


Evolution has exploited the rhythmic properties of 
neurons over hundreds of millions of years. For that 
reason, we should not expect to find only a single 
neural code. What we do know currently is that brain 
rhythms are very widespread and that they are associ¬ 
ated with known functions. 

Finally, waves can also interfere with each other. 
When you place a radio receiver next to a computer, 
you will hear a burst of noise whenever you press the 
keyboard. That is because each key press triggers an 
electromagnetic signal that radiates into the surround¬ 
ing space. Wave interference is a fundamental phe¬ 
nomenon in the physics of radiation. Interference may 
have important uses in the brain, but it might also 
degrade neural information processing. We are only 
beginning to understand the role of brain rhythms, 
but it is likely that wave interference will turn out to 
have observable effects as well. 


2*8 Gamma activity has multiple roles 

Figure 8.18 shows a classic study of gamma activity in 
visual perception, using direct brain recording of elec¬ 
trical activity on the cortex. Intracranial EEG (iEEG) has 
a far better signal-to-noise ratio than scalp EEG, since it 
is recorded from the brain itself. iEEG voltages typically 
are measured in millivolts; scalp EEG is measured in 
microvolts. For that reason iEEG has major advantages. 
The great drawback is that it requires invasive surgery. 
Animal studies often use intracranial electrodes, but 
human iEEG is limited to patients with a compelling 
medical need. Epilepsy surgeons use iEEG to localize 
seizure-triggering brain regions and for locating areas 
of the cortex that must be protected to avoid harming 
essential functions, like language and vision. Patients 
are conscious during this kind of brain surgery so that 
they can answer questions and report their experiences. 
The cortex itself does not have pain receptors, and local 
anesthetic at the site of the incision may be enough to 
minimize discomfort. Thus, the patient is normally 
conscious, even though his or her brain activity is con¬ 
stantly recorded and stimulated directly at the cortex 
(Crone, 2006; Canolty, 2006). 

Tallon-Baudry and Bertrand (1999) asked conscious 
patients with iEEG brain implants to report the pres¬ 
ence of a visual triangle. Initially only real triangles 
were presented, evoking the two major gamma-range 
bursts of activity marked with the yellow color in 
Figure 8.18. On the upper graph in column B, notice 
that two (yellow) gamma bursts appear immediately 
after stimulus presentation (time = 0). The first gamma 
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FIGURE 8.18 Evoked gamma bursts reflect visual feature binding. In a breakthrough experiment, 
Tallon-Baudry and Bertrand (1999) showed high gamma activity (yellow bursts) in the iEEG (direct brain 
recordings) of epileptic patients asked to report the appearance of a visual triangle. Two bursts were observed, 
one that occurred at stimulus presentation and a second that discriminated between triangle recognition and 
nonrecognition. Triangle recognition occurred with both the real triangle and the illusory triangle but not the 
'no-triangle' stimulus. Both gamma bursts plausibly involve conscious and unconscious aspects of vision. The 
silhouette of the head (c) shows the distribution of the second gamma burst over the occipital and some pari¬ 
etal cortex. Source : Tallon-Baudry and Bertrand, 1999. 


burst, around 50 to 150 ms, reflects perception of the 
visual input. The second gamma burst occurs about 
250 to 300 ms after stimulus onset and appears with 
both the real and the illusory triangle but not with 
control stimuli that do not look like a triangle. The 
illusory Kanisza triangle in column A of Figure 8.18 
has no sides, only three white pointed wedges against 
three dark disks. But subjects still perceive the illusory 
triangle as real. Thus, the second gamma burst signals 
a perceived triangle. It does not occur when a pseudo¬ 
triangle is presented (lower graph). The moment of 
recognition of the triangle can be considered as a 
match between the visual input and a memory. It is 
the visual cortex saying, 'There it is'! in response to the 
visual triangle. 


As you can see from the top of column C, the sec¬ 
ond gamma burst is located over the subject's early 
visual cortex. Because visual areas have two-way con¬ 
nections, it is likely that higher-level visual regions 
may also be oscillating in synchrony with the occipi¬ 
tal cortex (area VI). As we will see, one likely func¬ 
tion of gamma activity is to synchronize multiple 
visual maps. 

A series of iEEG studies over the past decade have 
demonstrated that gamma bursts (and other frequen¬ 
cies) occur in close association with cognitive tasks. 
They are an important part of the signaling mecha¬ 
nism underlying neural information processing. 
Because that has been a contentious subject in previ¬ 
ous years, the new evidence (along with novel EEG 
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(a) Passive listening 


(b) Active listening 




210ms 



(d) Auditory evoked components 



FIGURE 8.19 Gamma activity in listening. Auditory perception shows striking gamma band bursts 
in intracranial EEG (a), markedly stronger in the 'active listening' condition (b). This condition presumably 
adds executive attention to the conscious auditory stimulus. The term induced gamma (c) refers to endogenous 
brain activity that occurs with a 210-ms lag after the stimulus onset and may reflect reentrant activity between 
multiple levels of analysis (such as acoustics, speech phonemes, syllables, or morphemes and words). Source : 
Tallon-Baudry and Bertrand, 1999. 


and MEG findings) is very important. Notice that 
measuring fast oscillations with fMRI is more diffi¬ 
cult because of the five-second rise time of the blood- 
oxygen level (BOLD) response. Yet fMRI has its own 
great advantages, and converging studies of fMRI 
and electromagnetic brain signals have become very 
important. 

Figure 8.19 extends these findings to hearing. 
On the left side, passive listening results in induced 
gamma (about 200 ms after stimulus onset). On the 
right graph, the duration and frequency spread of the 
gamma activity are longer and more widespread in 
'active listening', a more attention-demanding way of 
listening. There are now many findings showing that 
active executive attention enhances gamma activity, as 
we will see. There is a great deal of psychological evi¬ 
dence showing that active listening improves stimulus 
processing and learning. 

Notice that these iEEG were published in the past 
decade. They represent a marked advance in our 
understanding of high-frequency events in the brain. 
While animal studies with implanted electrodes have 
been used for many years, these human studies have 
served to confirm and extend their findings to human 
cognition. Implanted iEEG in humans has much better 
spatial sensitivity than surface EEG, as well as better 


sensitivity and signal-to-noise ratio. A higher range of 
frequencies can also be recorded. Together with non- 
invasive brain imaging methods, these kinds of data 
are having a very great impact on our understanding 
of the brain in human cognition. 


2.9 Gamma synchrony may bind visual 
features into conscious objects 

So far, we have seen two kinds of evidence for gamma 
in cognition: gamma amplitude and gamma syn¬ 
chrony. Amplitude is the loudness of the rock band; 
synchrony is the degree to which everybody plays in 
rhythm. It is likely that the two go together. However, 
by high-temporal-resolution recording, gamma syn¬ 
chrony can be separated from gamma amplitude. 

One possible role of gamma synchrony is to 'bind' 
or integrate separate stimulus features into meaning¬ 
ful objects (see Figures 8.12 and 8.13). Different visual 
regions represent different features, and at some point 
the brain needs to bring them together into a single 
object. That may also be the point at which visual 
information becomes conscious. 

The object recognition area of the macaque brain 
has been shown to differentiate between conscious and 
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FIGURE 8.20 Face perception involves gamma activity in 
multiple visual maps. Lachaux et al. (2005) used iEEG to record dur¬ 
ing face perception directly from four different visuotopic regions in 
the human cortex. The stimulus was a "Mooney face" (top), which 
takes some time to recognize as a human face. Three of the four 
regions show a fast, intense gamma band burst beginning <100 ms 
after stimulus onset (from the top, fusiform gyrus, area VI, lateral 
occipital gyrus [BA19], and left intraparietal sulcus). The fourth, 
area VI, actually shows a decrease in gamma for about 700 ms (dark 
area), followed by a high gamma burst. The authors conclude that 
the gamma band response can be different in different brain regions. 
These differences may be due to selective inhibition in different vis¬ 
uotopic maps. Inhibitory neurons are believed to be constantly shap¬ 
ing the massive activity of excitatory pyramidal cells in cortex. The 
late gamma burst in VI may reflect top-down activation from the 
face recognition area (fusiform gyrus). Source: Lachaux et al., 2005. 


unconscious visual input. When we see a face, many 
visual feature maps in the brain show gamma activ¬ 
ity (Figure 8.20). Conscious objects combine multiple 
features; in a normal visual scene we do not see float¬ 
ing points of light or points of color or contrast edges. 
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TRENDS in Cognitive Sciences 


FIGURE 8.21 Conscious feature integration by gamma syn¬ 
chrony. Is feature integration required for conscious object percep¬ 
tion? Many scientists have argued for this position. Engel and Singer 
published a classical experiment in which the "dominant assembly" of 
neurons could be attributed separately to the left or right eye. Left or 
right visual dominance columns in cat cortex were monitored directly, 
using binocular rivalry for motion. A drifting grating moved in oppo¬ 
site directions in the two visual fields, providing two competing 
inputs that could not be fused into a single, coherent percept. Gamma 
activity synchronized only in the dominant visual field, which the cat 
was trained to match with an unambiguous stimulus, thereby signal¬ 
ing the conscious stimulus. Source: Engel and Singer, 2001. 


But we can see visual objects as distinct conscious 
experiences. 

Figure 8.20 does not directly show synchrony but 
only amplitude in the gamma range. One of the early 
experiments to demonstrate gamma synchrony between 
brain areas in conscious vision was published by Engel 
and Singer (Figure 8.21). 

Engel and Singer studied binocular rivalry in the 
cat (see Chapter 6), a classic paradigm that allows us 
to compare conscious versus unconscious stimulus 
processing in the visual brain with identical physical 
input to the two eyes. They recorded directly from 
ocular dominance columns in visual cortex, which 
distinguish between the input from the right and 
left eye. Experimental cats were surgically altered to 
have nonoverlapping visual fields, a condition called 
strabismus or Tazy eye' when it happens naturally. 
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Strabismus leads to one-sided ocular dominance in the 
visual cortex, determining which eye will dominate 
vision; that is, which eye 'is conscious'. In this case, 
when the rivaling stimulus became conscious in the 
right versus left eye, the dominant cortical field could 
be recorded separately for each eye. 

Engel and Singer (2001) showed two opposing 
streams of moving grids to each of the cat's eyes, a 
kind of binocular rivalry that cannot be fused into a 
single, conscious visual experience. The results are 
shown in Figure 8.21. Notice that in the cat's cortex, 
the dominant (conscious) eye demonstrated gamma 
synchrony locked to the 'conscious eye', whereas 
the nondominant (unconscious) eye showed no syn¬ 
chrony. The cat was asked, in effect, what it saw, by 
stimulus-matching to an unambiguous drifting grat¬ 
ing, as if the experimenters were asking, 'Are you see¬ 
ing this one or the other one'? Thus, the cat was able 
to provide a 'voluntary report' on the conscious stim¬ 
ulus. Similar experiments have been extensively per¬ 
formed in macaques, with comparable results. 

Notice that the peak level of activity in Figure 8.21 
occurs with perfect synchrony at time t = 0 in the domi¬ 
nant assembly of neural populations. A smaller peak 
occurred at time t = + / — 20 ms and at time multiples 
of 20 ms. These peaks fall off as the visuotopical neural 
maps drift away from perfect synchrony. Nevertheless, 
even the smaller peaks occur at 20 ms intervals, suggest¬ 
ing that gamma synchrony is occurring at 50 Hz - that 
is, one cycle per 20 ms. In contrast to the dominant neu¬ 
ral assembly, the suppressed assembly shows little or no 
synchrony. This was also the stimulus that the cat could 
not match to the sample stimulus, as if it were telling us 
that there was 'nothing to see' in the nondominant eye. 

The Engel-Singer type of experiment involves an his¬ 
toric advance in our understanding of conscious vision 
in other mammals. Similar experiments have been per¬ 
formed in humans as well, as discussed in Chapter 6. 

A very different kind of method provides converg¬ 
ing evidence for the role of brain rhythms in con¬ 
scious vision. Figure 8.22 shows a recent finding using 
transcranial electrical brain stimulation, specifically a 
technique using an alternating-current (AC) field. The 
graph is a traditional psychophysical curve, relating 
a physical stimulus to a subjective experience. In this 
case, the conscious experiences were 'phospenes', visual 
'flashes' that can be evoked by direct brain stimulation. 
Phosphenes were first demonstrated in humans by the 
pioneering neurosurgeon Wilder Penfield and colleagues 
some 50 years ago during presurgical exploratory stud¬ 
ies of intractable epilepsy patients (Penfield & Roberts, 
1959). However, the conditions for reliable phosphenes 
have not been well understood until recently. Because 


Artificially induced rhythm - occipital 



FIGURE 8.22 Transcranial stimulation can evoke conscious 
"flashes'. Conscious phosphenes (visual flashes) can be evoked 
across the scalp by transcranial Alternating Current Stimulation 
(tACS) over the occipital cortex. Kanai et al. (2008) used tACS to 
sample the frequency spectrum from 3 to 30 Hz and obtained these 
elegant psychophysical curves relating tACS frequency to subjec¬ 
tive visual sensations. The curve peaks just below 20 Hz in the beta 
range, suggesting that intrinsic rhythms were induced by the elec¬ 
trical field. Source : Kanai et al, 2008. 

they are subjective experiences (unlike other types of 
direct stimulation of the visual cortex), they clearly tell 
us something about visual consciousness. 

A recent study by Kanai et al. (2008; Figure 8.22) 
shows reliable phosphenes evoked by alternating cur¬ 
rent in the beta range (—20 Hz, sometimes called low 
gamma). This finding is again quite historic. The psy¬ 
chophysical function obtained by Kanai et al. demon¬ 
strated how regular these phenomena now appear to 
be. This is significant, because phosphenes long have 
been considered to be difficult to evoke in a replica¬ 
ble way. The consistent peak just below 20 Hz is also 
remarkable and suggests that conscious visual events 
require rhythmic brain events, as we would expect 
from the iEEG experiments just presented. Notice that 
the peak subjective rating rises predictably as the cur¬ 
rent level is increased (in micro-amperes at the scalp). 

If the evidence presented so far makes the case for 
gamma synchrony as one ingredient of common cog¬ 
nitive tasks, how does gamma synchrony 'run' a com¬ 
plex task? After all, perceiving a face or a word takes 
a lot of processing. It is performed very fast, and one 
component of the task needs to rapidly lead to the fol¬ 
lowing one. Independent evidence suggests, for exam¬ 
ple, that the purely sensory aspect of perception may 
occur before the analysis of meaning, at least in lan¬ 
guage perception. The next experiments shows how 
successive spurts of gamma synchrony may enable 
different phases of a single cognitive task. 

Again, robust evidence has been found in explora¬ 
tory studies of conscious epileptic patients using iEEG 

















2.0 WAKING 


267 


Task: Hearing a word 
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Task: Repeating a word 


FIGURE 8.23 Multiple synchronous gamma bursts in speaking and hearing. Crone et al. (2006) studied conscious epileptic patients 
with implanted cranial electrodes. The results suggest that the brain is triggering synchronous gamma bursts between different brain regions 
involved in hearing, speech perception, and speech production. Each phase of the name repetition task seems to evoke a separate synchronous 
gamma burst in somewhat different regions of the left hemisphere in a single patient. Gamma synchrony was observed in four parts of the 
high gamma band, ranging from 57 to 117Hz (see pink, blue, and purple arrows). Red horizontal arrows point to gamma synchrony observed 
in specific parts of the frequency range. In the graphs, red colors indicate highest amplitude, with yellow and light blue colors representing 
lower amplitudes. The graphs show frequency on the vertical axis and time before and after the stimulus in hundreds of milliseconds on the 
horizontal axis. Source : Adapted from Crone et al., 2006. 


to localize the epileptic source and to identify those 
brain regions whose damage during surgery would 
affect essential functions like vision, audition, and lan¬ 
guage. Crone et al. (2006) demonstrated that gamma 
synchrony between local regions of the left hemi¬ 
sphere is often transient , indicating a brief period of 
coordination between two parts of the brain. In Figure 
8.23, a patient with a grid of electrodes over the left 
hemisphere showed event-related gamma synchrony, 
with brain rhythms triggered by a simple picture 
naming task. The patient was given a series of spoken 
words and asked to keep each one in memory and 
then repeat it on cue. In this way, auditory input and 
the naming response could be kept separated in time. 


The purple colors in the figure show different bands 
in the high gamma range (67-117 Hz) in different parts 
of the left hemisphere. Hearing each word triggered 
several synchronous gamma bursts over the audi¬ 
tory and speech perception cortex (Wernicke's area). 
Speaking the same word triggered gamma synchrony 
over Broca's area, as we might expect. Notice that 
local cortical patches of the left hemisphere showed 
synchronous bursts in a complex series, as indicated 
by the purple, pink, and blue arrows. 

So far we have seen local gamma synchrony in local 
areas of cortex. Long-range gamma synchrony also has 
been observed, as we will see later, among distant parts of 
the cortex that must cooperate in carrying out some task. 





















































268 


8. CONSCIOUSNESS AND ATTENTION 


2.10 Theta rhythms have multiple roles 

Gamma rhythms are not the only important oscilla¬ 
tions in the brain; for purely historical reasons, gamma 
has come to mean a wide, high band of oscillations 
from perhaps 30 to 120 Hz and sometimes higher. This 
is an accident of scientific history, simply because fast 
oscillations were more difficult to record at one time. 


in part because they show up better with implanted 
electrodes. It is clear, however, that slower rhythms 
play fundamental roles. The theta rhythm is one of the 
slowest oscillations in the normal waking state, cycling 
4 to 7 Hz, just above the delta rhythm that dominates 
SWS, and the more recently discovered slow oscilla¬ 
tions that may go down to .01 Hz (Figure 8.24). 



(b) 


(c) 


*• 
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• . • 


passive listening to predictable tones 


2-back phoneme working memory 



FIGURE 8.24 Theta-gamma multiplexing across 16 different tasks. Theta-gamma phase 
locking across 16 different tasks, using intracranial EEG (iEEG). (a) Shows an index of phase 
locking (t = 0 indicates precise phase locking, with less correlation as the time lag increases in 
either direction). The left lateral hemispheres shown for Subject 2 show colored dots for elec¬ 
trode locations corresponding to the color bar in the center of (b) and (c). Notice that passive 
listening to predictable tones shows theta-gamma locking (d), but, as shown in (c), a difficult 
working memory task, reporting two phonemes back from the current phoneme, has more yel¬ 
low and red colors, indicate increased phase locking in the more demanding task, (d) Shows 16 
tasks, with the highest correlations (red) for squares nearest the diagonal, which are the most 
similar to each other. The same point is made in (e), indicating low but significant correlations in 
phase locking between similar tasks compared with dissimilar ones. Source : Canolty et at., 2006. 
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Theta seems to 'carry' gamma oscillations in much 
the way an AM radio frequency carries a voice sig¬ 
nal. Theta is believed to play a key role in episodic 
memory and frontal lobe activities. Theta is one of 
the basic carrier frequencies of the hippocampus, 
observable during both episodic memory encod¬ 
ing and recall. Theta is believed to enable the coding 
and decoding of hippocampal learning in the neocor¬ 
tex, especially the frontal lobes. In the surface EEG, it 
can be observed along the midline of the scalp when 
episodic memories are recalled and during executive 
tasks that engage both the lateral and medial frontal 
lobes. 

There are many ways for brain waves to interact: 
like water waves, they can add on to each other or can¬ 
cel each other. Frequency modulation is also possible, 
allowing for speeding and slowing of waves. What's 
more, there is evidence for phase modulation, so that 
the degree of phase locking between two sources of 
gamma might shift in order to pass some signal. Phase 
modulation has been proposed as a mechanism for hip¬ 
pocampal coding of place memory in rats. 

There is no reason to suppose that the brain has 
only one or two kinds of population codes. The nature 
of neural coding is a hot scientific question, and it is 
by no means settled. 

We have seen that theta oscillations may group 
gamma, and in some conditions alpha may group 
gamma activity as well. Steriade (2006) has suggested 
that slow oscillations may generally work to group 
faster ones. Even the newly discovered 'slow oscilla¬ 
tions' that range from 0.1 to the delta range may serve 
to group theta, alpha, beta, and gamma. 


2*11 Alpha rhythms 

In 1929, a German psychiatrist named Hans Berger 
made the first observation of human EEG by placing 
an electrode on his young son's head just above the 
occipital cortex. The key to Berger's discovery was a 
new, powerful amplifier that could amplify the small 
electrical voltages on the surface of the head and con¬ 
vert them into visible deviations of an ink pen, writ¬ 
ing traces on a moving strip of paper. Berger asked 
his son to open and close his eyes and found that with 
eyes closed, a small sinusoidal wave could be picked 
up. Berger called it the alpha wave (Figure 8.25). (The 
names alpha, beta, and gamma, etc., are largely histori¬ 
cal accidents.) Alpha waves conventionally are defined 
to oscillate from 8 and 12 Hz (Hertz = cycles per sec¬ 
ond). The size of alpha has been found to be increased 
by internal tasks, such as mental calculation and work¬ 
ing memory. Recent theory suggests that alpha reflects 
the selective suppression of task-irrelevant areas and 
activities in the brain (Palva and Palva, 2007). 

In retrospect, Berger's discovery was a historic 
moment in brain science and medicine. But until the 
last few decades, the surface EEG was difficult to ana¬ 
lyze. Even today, scalp EEG poses practical difficulties. 
For example, we still have no foolproof way of locat¬ 
ing the sources of the surface potentials. That problem 
can be solved with other techniques, however, includ¬ 
ing intracranial EEG, MEG, PET, and fMRI. Surface 
EEG has important practical applications in medicine. 
It is by far the easiest and least expensive brain record¬ 
ing method, and it continues to be useful. There are 
constant efforts to improve its accuracy. 


Wakefulness (eyes open) 



Wakefulness (eyes closed) 1 * 

FIGURE 8.25 Alpha waves with eyes open and closed. The raw EEG over the occipital cortex, showing irregular wak¬ 
ing activity with the eyes open (upper trace) and spontaneous alpha waves with eyes closed (lower trace). A great deal of 
research has been conducted since Berger's 1929 discovery of these phenomena, but they are still something of a puzzle. 
One major question is what occurs to irregular waking EEG when the eyes are closed, and alpha 'idling' appears instead. 
Alpha is then presumably actively suppressed, just as complex, waking EEG appears to be suppressed when alpha appears 
in the raw, unanalyzed EEG. Source : Tononi in Laureys & Tononi, 2009. 
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The brain is full of criss-crossing electromagnetic 
waves. What is remarkable about Berger's eyes-closed 
alpha waves is that they are so simple. Visible alpha 
is remarkably regular, easily observable, and related 
to cognitive events. The traditional explanation is that 
the brain is in an 'idling mode'. 

Berger's next finding was called 'alpha blocking', 
and it is also straightforward. When subjects gener¬ 
ate regular alpha waves they will be interrupted when 
they open their eyes or perform active thinking or 
imagery. Instead of regular sine-like waves, the EEG 
over the occipital cortex shows irregular, fast, and 
low-voltage waves (see Figure 8.4). These fast waves 
were called beta (13-30 Hz) and gamma waves (30 Hz 
and higher). The beta-gamma signal is a useful sign 
of conscious waking activity. It is routinely used for 
medical diagnosis. Keep in mind, however, that the 
visible EEG waves are just the waves on top of the 
oscillatory lake. Underneath the irregular scalp EEG 
there are known to be synchronized rhythms. They 
can be found by mathematical analysis and by record¬ 
ing with deep-brain electrodes. It therefore seemed as 
if there were two kinds of EEG signals: alpha waves, 
sometimes considered to reflect 'mental idling', and 
active thinking, marked by complex, fast, and irregu¬ 
lar brain activity. Considering the complexity of the 
brain, these early observations were surprisingly sim¬ 
ple and lasting. 

Sinusoidal waves are very common in nature. They 
can be observed when any rotating source sends out 
a signal over time. If you attach a small light to the 
rim of a bicycle wheel and ride the bike in a straight 
line at night, the light will trace out a sine wave. The 
mathematician Joseph Fourier (1768-1830) proved 
that any complex wave can be analyzed into a set of 
sine waves, a process that is called Fourier analysis. 
Modern EEG is routinely run through high-speed 
Fourier analysis, so that we can see the sine-wave 
components of complex brain activity. In prac¬ 
tice, we often tend to group wave components into 
bands , using the conventional ranges, like 4 to 7 Hz 
for theta, 8 to 12 Hz for alpha, and so on. However, 
it is important to bear in mind that nature may not 
always go along with our conventions. Different sub¬ 
bands of gamma routinely seem to do different tasks, 
for example. 

Recent studies show that alpha can be found during 
mental imagery and other mental activities (Figure 8.26). 
Like other slow rhythms, alpha may also group faster 
activities. Intracranial EEG recordings show both alpha 
synchrony and desynchrony. (Desynchrony is similar to 
Berger's observation of 'alpha blocking'.) 


Memory set Probe 



Time (s) 


FIGURE 8.26 Alpha may group gamma oscillations in work¬ 
ing memory. In an immediate memory task, Palva and Palva (2007) 
showed a strong burst of alpha activity (the horizontal red and 
yellow bar) for three seconds while subjects hold a number in mem¬ 
ory Notice that the most intense activity is consistently centered 
near 12 Hz. Source : Palva and Palva, 2007. 

We are only beginning to understand neural cod¬ 
ing in the brain. It is important to keep in mind that 
both synchrony and its absence may convey informa¬ 
tion, in much the way that both the presence and the 
absence of light reveals visual information - patterns 
of shadows are as useful as patterns of light. Signaling 
codes use both signals and the absence of signals, as 
in Morse code. 

Palva and Palva (2007) suggest that cross-frequency 
phase synchrony between alpha, beta, and gamma 
oscillations 'coordinates the selection and mainte¬ 
nance of neuronal object representations during work¬ 
ing memory, perception, and consciousness'. In some 
experimental conditions both alpha synchrony and 
desynchrony can be observed. 

Figure 8.26 is a clear demonstration of the involve¬ 
ment of alpha activity while holding items in work¬ 
ing memory. As mentioned previously, it is likely that 
this increase in alpha activity may also be multiplexed 
with other waveforms. But the evidence suggests that 
alpha plays a functional role in human cognition and 
that it does not represent only an 'idling rhythm', as 
many scientists believed until recently. 


3.0 ATTENTION ENHANCES 
PERCEPTION, COGNITION, AND 
LEARNING 

Common sense makes a distinction between attention 
and consciousness. The word attention seems to imply 
the ability to direct cognitive resources to some event. 






3.0 ATTENTION ENHANCES PERCEPTION, COGNITION, AND LEARNING 


271 



500 ms 

Time post-onset 


FIGURE 8.27 Bottom-up attention: A monkey threat face. 
Visual face-sensitive neurons fire faster when macaques see a threat 
face, as in the photo, compared with a neutral face. A threat face is 
a biologically important signal. The graph shows spikes per sec¬ 
ond in a neuron located in the visual face area, the fusiform gyrus. 
The threat firing rate is shown in red and the neutral rate, in black. 
Sources : Top, Parr and Heintz, 2008; bottom, Vuilleumier, 2005. 


If you can't remember a technical term in this book, 
for example, you might try to clear your mind of other 
thoughts and 'pay more attention' to trying to remem¬ 
ber the word. The word attention has a kind of 'point¬ 
ing' or 'directive' sense. But in everyday language, 
'consciousness' refers to an experience - of reading 
this sentence, for example, or of having thoughts, feel¬ 
ings, and images that may come up after reading these 
words. Selective attention implies a choice among pos¬ 
sible events. Consciousness seems to be the experience 
of an event after it has been selected. 

We can often decide what to become conscious of: 
whether to continue reading this book, turn on the tel¬ 
evision, stop thinking about a friend, and so on. There 
is also automatic attentional selection if stimuli are 
intense, dynamic, or biologically or personally impor¬ 
tant (Hahn et al., 2006). We constantly scan the world, 
even without intending to do so, and often pick up 
significant events automatically. 

Like most mammals, monkeys can be fierce fight¬ 
ers. When facing a threatening snake or competitor 
they tend to send out warnings first, using threat faces, 
screams, and behavioral displays. Figure 8.27 shows a 
'bared teeth' threat face made by a macaque. Monkeys 
and humans show increased responding in visual face 
neurons to threat faces. They are biologically vital. For 
that reason, threat faces are likely to be processed 'bot¬ 
tom up', much as we expect alarm bells to be processed 


bottom up: This important stimulus interrupts the flow 
of other experiences. 

Young mammals can tell the difference between 
'pretend' aggression and the real thing. Play aggres¬ 
sion is part of 'rough and tumble play', something 
that young mammals often like. Lin et al. (2009) have 
shown that attention to threat faces can occur uncon¬ 
sciously, although the outcome of that attentional proc¬ 
ess is usually conscious. 


3*1 The Posner flanker task 

Posner and colleagues have devised a simple probe 
for visual attention called the flanker task. They asked 
subjects to pay attention to a stimulus at a known 
location on the right or left side of the fixation point 
(marked with a dot or a plus sign). Because humans 
have a very limited foveal 'keyhole' through which 
we fixate a small part of the visual field at any single 
moment, it is possible to control the exact informa¬ 
tion that is available to a subject. (We see only about 
3-6 degrees of visual arc when the eyes are fixed on a 
point. Try it!) 

The flanker task allows for testing of both volun¬ 
tary and nonvoluntary attention, by giving subjects 
either correct or incorrect information about the flank 
on which the target stimulus will appear (Figure 8.28). 
This can be varied from trial to trial. By mixing cor¬ 
rect and incorrect cues before a stimulus, data can be 
collected in a single experiment on expectation-driven 
(correctly cued) and unexpected (incorrectly cued) 
flankers. The task is simple enough to administer in 
an fMRI scanner in a half hour or so, and the resulting 
brain scans provide separate information about expec¬ 
tation-driven trials and unexpected trials. By subtract¬ 
ing the 'unexpected attention' brain activity from the 
'expected attention' scans, Posner and coworkers were 
able to obtain a relatively pure measure of the brain 
regions involved in voluntary visual attention. 

Because the flanker task is the most common 
method for studying visual attention, we will go 
through it in some detail. The flanker task is simple, 
effective, and adaptable. For example, the target stim¬ 
uli can be emotional faces, allowing us to explore how 
the brain pays attention to emotional events (Fan et al., 
2002; Posner and Petersen, 1990). 

Here is how the flanker task works: 

1 Subjects keep their gaze on the fixation point. They 
look at the target from a known distance so that 
the degree of visual arc subtended by the fixation 
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(a) 



The arrow points away from the target location 


FIGURE 8.28 The flanker task for assessing attention. The flanker task is used to study vis¬ 
ual selective attention and its brain basis. The subject looks only at the fixation point. Directional 
cues like the arrow draw attention to left or right flank, but no eye movements are allowed. 
Accuracy and response speed measure costs and benefits, depending on whether the flanker was 
cued or counter-cued. Targets presented on the unexpected side have a cost in time and accuracy 
and are believed to reflect the ability to override cued attentional set. Source : Reynolds et al., 2003. 


point is always the same. The fovea (the highest- 
density region of the retina) is kept focused on the 
fixation points, and data from trials with eye move¬ 
ments away from the fixation points typically are 
discarded. Outside of the fovea, which covers only 
2 to 4 degrees of arc, the retina loses resolution and 
is sensitive only to light and dark edges. (It follows 
that our normal sense of a rich and colorful visual 
world is a construction of the visual brain, not a lit¬ 
eral record of the input into the retina.) 

2 When flanking stimuli appear on the right or left 
side of the fixation point, they can be detected using 
only covert attention, because the eyes are kept 
fixed on the crosshairs. Posner and Petersen (1990) 


discovered that people can selectively increase the 
attentional processing efficiency of retinal inputs 
that are a few degrees off the fovea (see Chapter 6). 

3 Subjects may be cued by an arrow pointing left or 
right to expect a target to flash briefly to the right 
or left of the crosshairs, or a momentary brighten¬ 
ing of the left flanker box tells the subject that the 
target will soon flash on the left side with 80% 
probability. 

4 In the figure, the target is flashed in the expected 
location for a fraction of a second. Subjects respond 
as quickly as possible. When their cued expecta¬ 
tions are correct, their reaction times and accuracy 
are optimal. 
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5 However, the target might also be flashed to the 
other flank, in the unexpected location. This unex¬ 
pected event imposes a cost in reaction time and 
accuracy. 

Thus, attention has both benefits and costs. An increase 
in speed and accuracy at the expected location com¬ 
pared with an unexpected location is considered to be 
a benefit of attention. A loss of speed and accuracy for 
the unexpected stimulus is considered to be the cost of 
the failing to pay attention to that event. The flanker 
task therefore permits quantitative assessment of the 
cost-benefit tradeoffs in terms of speed and accuracy. 

The ANT task (Fan et ah, 2002) is a generalized ver¬ 
sion of the flanker task, to allow testing of three sepa¬ 
rable aspects of attention: alerting before an expected 
signal (after an 'alert' cue), orienting to a specific loca¬ 
tion in space (using an arrow to point to the coming 


flanker stimulus), and executive attention, to act against 
expectations set up by the task. In this paradigm, 'exec¬ 
utive attention' is taken to be the opposite of 'automati¬ 
cally controlled attention'. The Stroop Task is a good 
example of executive or voluntary attention (Box 8.1). 

The ANT task allows each component to occur at 
a different moment than the others. That fact allows 
brain images to be recorded for the three different 
components. Various task manipulations can be used 
to selectively change alerting, orienting, and executive 
attention. For example, asking subjects to keep sev¬ 
eral numbers in memory during the task may degrade 
executive attention because it makes use of central 
limited capacity but not automatic attention. 

Consider the case of a college student in a lecture 
room, with many sensory inputs occurring at the same 
time. The student must be reasonably alert, orient to 
the desired sensory stream, and allocate attentional 


BOX 8.1 Stroop color-naming and executive attention 


Equipped with a stopwatch and Fig. 8.29, you can easily 
replicate Stroop's original demonstration (Stroop, 1935). 
For columns 1 and 2, the task is to read each list of words 
aloud as fast as possible, ignoring their print color. Begin 
by covering all of the columns except column 1. Start the 
timer when you say the first word and stop it when you 
say the last word. Record your time. Now cover all col¬ 
umns except column 2 and read the words aloud again. 
For columns 3 and 4, the task is changed to naming the 
print colors aloud as fast as possible, ignoring the letters 
or words. Do this for the rows of colored letters in column 
3 and then for the rows of colored words in column 4. 

Stroop observed three primary results. First, read¬ 
ing words was faster than naming colors. This is consis¬ 
tent with word reading being more practiced and hence 
more automatic than color naming (Cattell, 1886, and 
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FIGURE 8.29 The Stroop task. 


Fraisse, 1969). Second, there was little difference in read¬ 
ing the words in columns 1 and 2 (Stroop's Experiment 
1): mismatched ink colors did not produce interference 
in reading the words. Third, in sharp contrast, switch¬ 
ing from nonwords to words in color naming made a 
very large difference between columns 3 and 4 (Stroop's 
Experiment 2): naming the colors of incompatible color 
words showed dramatic interference. Apparently, the 
greater automaticity of word reading leads to the words 
being read even though they should not be, producing 
conflicting responses to each stimulus. This both slows 
down color naming responses and makes errors - reading 
words instead of naming colors - more likely. 

Modern versions of the Stroop task are typically com¬ 
puter-controlled displays of a single word in color, rather 
than multiple items on a card, permitting more control 
and more precise measurement of individual item and 
sequence effects. Also, although Stroop did not include a 
congruent condition (RED printed in red; say Ted'), mod¬ 
ern versions of the task often do. Both of these modifica¬ 
tions were introduced by Dalrymple-Alford and Budayr 
(Dalrymple-Alford & Budayr, 1966). 
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FIGURE 8.30 Posner attentional control network. Using the flanker task, Posner and colleagues demon¬ 
strated activation of a network of brain regions for executive attention. The image on the right shows high tha¬ 
lamic activation, probably involving the pulvinar and mediodorsal nuclei of the thalamus. See Figure 8.31 for a 
model using these areas. Source : Fan et at., 2005. 


resources to the task of keeping track of the lecture, 
taking notes, and the like. Brain areas for alertness are 
located in right frontal and parietal regions (Witte and 
Marrocco, 1997). But as we have seen, alertness also 
requires the waking state, which depends on the brain¬ 
stem, thalamus, and cortex. Brain areas for orienting 
are located in right temporoparietal junction (TPJ) and 
the inferior frontal gyrus (IFG, corresponding to Broca's 
area in the language hemisphere) (Corbetta et al., 2000). 
Finally, voluntary control of attention involves the 
anterior cingulate and lateral prefrontal cortex (ACC 
and L-PFC), the pulvinar nucleus of the thalamus, and 
the superior colliculus (SC) (Marrocco and Davidson, 
1998). (See Figure 8.30.) All of these regions have visuo- 
topical maps of one kind or another. 

We do not want to confuse selective attention with 
eye movements, which also enhance the quality of vis¬ 
ual input. Posner and Petersen (1990) discovered that 
attentional shifts can be accomplished while keep¬ 
ing the eyes fixed on a single point. That is, we can 
enhance processing a few degrees to the right or left 
of our visual fixation point, even without moving our 
eyes. Visual attention experiments routinely use such 
implicit attention conditions. 

Visuotopic neurons respond to optical stimuli at 
different levels of analysis (see Chapter 6; Figure 8.31; 
Itti and Koch). Figure 8.31 gives us a convenient over¬ 
view. Each layer of the pictured model by Itti and 
Koch responds to a particular feature of the stimulus: 
color, line orientation, contrast, and object identity. 
This is a simplification of the visual brain, which is far 
more complex and flexible and which must deal with 
complications such as the constant motion of the eyes 
and the head, the very narrow limits on foveal vision, 
and much more. But Figure 8.31 helps to clarify our 
question. 


Each visuotopic map is a two-dimensional mosaic 
of neurons with rather narrow receptive fields (see 
Chapter 6). We can therefore ask a more focused ques¬ 
tion: does attention to some event or to some location 
enhance signal processing in the appropriate map and 
the correct receptive field? If the watcher in Figure 
8.31 is hot and thirsty while wandering in the Sahara 
Desert, will his or her attentional system enhance 
visual processing for ice-cold mugs of beer located on 
the left side of his visual field? This question is much 
more specific and testable. 

3*2 A model of attention 

Itti and Koch (2002) developed a useful model of 
attention that combines a number of important fea¬ 
tures. It shows a simplified layered concept of the 
visual system, with multiple topographic visual maps. 
The visual maps show line orientation, stimulus inten¬ 
sity (contrast), color, and salience. 'Salience' is defined 
in terms of feature contrast in any visual map. In light- 
sensitive regions, it is the contrast between light and 
dark patches on the map. In motion-sensitive areas 
like area MT, it may be a stable object against the 
background of a waterfall. A combined salience map 
may combine all the contrasting features of multiple 
visual arrays into a single overall saliency map, one 
that reflects the unusual, unexpected, or notewor¬ 
thy features of a visual scene at any level of analysis. 
A 'Winner-Take-All' (WTA) computation selects the 
most salient location on a combined map and inhib¬ 
its competing locations. Obviously, salience can be 
important to know, but it can also be misleading; for 
example, when you are watching a visually exciting 
music video that contains a variety of attention-driv¬ 
ing features, you may want to think about something 
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FIGURE 8.31 Multiple visuotopic maps, salience and learn¬ 
ing. Multiple visuotopic maps support our conscious experiences 
of beer mugs and deserts. Notice that the person at the bottom 
seems to be looking for a cold drink on a hot day in the Sahara. 
How do our brains select the cold beer? What do we become con¬ 
scious of? Source : Peter Thier et al., (2002); figure adapted from Itti 
and Koch, 2001. 


else. You may have to override what is most perceptu¬ 
ally salient at any moment. 

The concept of a saliency map reflects significance, 
motivational relevance, and vividness of the input. 
Many topographical maps in the visual brain are sen¬ 
sitive to motivation and relevance. The man at the 
bottom of the figure is imagined standing in a hot 
desert with a cold mug of beer on the far left side of 
his visual field - just outside of his direct visual field. 
Selective attention allows significant stimuli like the 
cold beer to emerge into consciousness. These can be 



FIGURE 8.32 A Winner-Take-All network. A Winner-Take-All 
(WTA) network, in which the most activated point in the horizon¬ 
tal plane map inhibits all the surrounding points. The vertical axis is 
labeled activity and may represent a single point in visuotopic space 
or the summed activity of multiple visuotopic maps. WTA networks 
are very commonly used in decision-making neural nets, and in the 
brain both selective attention and conscious perception may make 
their final "decisions' using a WTA mechanism. In the case of ambigu¬ 
ous stimuli or dual tasks, like binocular rivalry, the brain makes one of 
two competing interpretations conscious. One way to model that is to 
use a WTA network. In the case of ocular dominance columns in the 
visual cortex (above), we can directly measure a large cortical region 
that behaves that way when the visual system is provided with differ¬ 
ent inputs to the two eyes (Section 2.11). Source : Standage et ah, 2005. 


expressed as Top-down attentional biases' that alter 
the saliency map on top of the stack. Prior learning 
also is simplified as an input into the saliency map. 
All the topographical maps resonate together in syn¬ 
chrony and jointly take decisions in cases where the 
corresponding points on the majority of maps lead 
to the same overall result. The output may be an eye 
movement, allowing the viewer to see the cold beer 
mug, or it may be a covert shift in attention to the 
left, again allowing the beer stein to come to visual 
consciousness. Again, it is possible that the man may 
want to override the perception of the cold beer on 
a hot day and focus attention on crossing the desert 
instead. There are potentially competing decisions in 
this kind of multilayered network. 

An important aspect of the Itti-Koch attention 
model is the Winner-Take-All (WTA) network (Figure 
8.32). WTA networks essentially allow the most active 
point on the join topographical maps to 'win', with 
input from saliency, which represents such things as 
motivation, hunger, and thirst; prior learning about 
relevance; and so on. WTA is especially important 
because it also suggests an explanation for conscious 
experiences of ambiguous inputs, as in the case of 
ambiguous figures or binocular rivalry. Conscious 
experiences are marked by internal consistency, even 
when sensory inputs are not. Most of the words in the 
English lexicon are highly ambiguous, for example. 
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but in context (as in this sentence), ambiguous words 
are consciously experienced in terms of just one inter¬ 
pretation. Thus, a WTA network may be involved in 
an attentional system, as shown in Figure 8.25, but 
they are also a very powerful feature of conscious per¬ 
ception. Indeed, we can consider conscious perception 
to be the outcome of many attentional acts. In reality, 
there may be no difference in the brain between those 
two mechanisms. 

The common-sense distinction between attention 
and consciousness suggests that there are attentional 
control mechanisms that often determine what will or 
will not become conscious. This belief is backed by 
good evidence. 

The term attention is used most intuitively when 
there is a clear voluntary or executive aspect. We ask 
people to pay attention to things, which implies they 
can choose to do so or not, depending on some execu¬ 
tive decision-making processes. The easiest concept of 
attention therefore is voluntary selection of informa¬ 
tion. In human experiments, we invariably ask par¬ 
ticipants to pay attention to the stimulus conditions of 
the experiment. Voluntary attention is the kind that is 
studied most often, and as you might guess from the 
other chapters, it is likely to use prefrontal cortex in 
humans (see Chapter 12). 

Corbetta et al. (2002) recently wrote that volun¬ 
tary attention, 'is involved in preparing and apply¬ 
ing goal-directed (top-down) selection for stimuli and 
responses'. Automatic attention, on the other hand, 
'is not involved in top-down selection. Instead, this 
system is specialized for the detection of behaviorally 
relevant stimuli, particularly when they are salient or 
unexpected'. 

While the two systems overlap, they provide a use¬ 
ful first take on the brain basis of selective attention. 

However, when we hear a sudden loud noise, our 
attention is 'captured', even without executive guid¬ 
ance. Attentional capture is an important topic, as we 
will see in the next section on stimulus-driven atten¬ 
tion. As you might expect, visual attention can be 
captured by human faces, emotional expressions, and 
bodies, when compared with neutral stimuli. Intense 
or sudden stimuli, or unexpected events in general, 
generate larger brain responses than control stimuli. 
Thus, we can talk about 'bottom-up' capture of selec¬ 
tive attention, driven by stimuli, as well as top-down, 
goal-driven attention, under executive guidance. 

In the real world, voluntary and automatic atten¬ 
tion are generally mixed. We can train ourselves to pay 
attention to the sound of the telephone ringing. When 
it rings and we suddenly pay attention to it, is that 


voluntary or automatic? Well, it began being voluntary 
and became more automatic. The dimension of volun¬ 
tary versus automatic attention is therefore a contin¬ 
uum. Perhaps the strongest case of voluntary attention 
is the one where we must exert intense mental effort 
over a period of time (see Box 8.1). A clear example of 
the opposite pole of the continuum might be a case of 
a loud sound or a biologically important event like a 
crying baby, which is hard not to pay attention to. 

In sum, attention is defined here as the ability to 
select information for cognitive purposes. Selection may 
be shaped by emotion, motivation, and salience, and is 
at least partly under executive control. Thus, selective 
attention works closely with all the other components 
of our framework diagram (chapter-opening figure). 
Without flexible, voluntary access control, human 
beings could not deal with unexpected emergencies or 
opportunities. We would be unable to resist automatic 
tendencies when they became outdated or change atten¬ 
tional habits to take advantage of new opportunities. 

On the other hand, without stimulus-driven atten¬ 
tion, we would be slow in selecting significant events. 
We need both voluntary and automatic attention. 

As we have seen, specific regions of the brain sup¬ 
port functions like sensory perception, working mem¬ 
ory, language, and long-term memory. The same is true 
of selective attention. As techniques have focused on 
smaller brain areas working at faster speeds, we have 
learned more about attentional networks and how they 
select specific kinds of sensory information. Many lines 
of evidence now agree on some basic facts. 

As Figure 8.33 shows, attention is believed to be a 
selective capacity, either under voluntary control or 
driven by a stimulus. The result of selective attention 
is to enhance the quality of the selected information or 
at least to try to do so. What is the evidence for atten¬ 
tional enhancement? 

Voluntary attention in the brain has two aspects, as 
emphasized by the large yellow arrow in Figure 8.33. 
Frontal and parietal regions are thought to be involved 
in directing voluntary attention (labeled in blue print); 
in the case of vision, the target regions are thought to 
be in the visual cortex (labeled in red print). Thus, two 
questions for a model of attention have to do with 
(a) the control of selectivity, and (b) the properties of 
the target that are being attended. 

Figure 8.34 shows a current set of hypotheses about 
specific brain regions involved in voluntary attention 
to a visual location or stimulus. Notice that voluntary 
control of attention is attributed to the prefrontal cor¬ 
tex. Top-down activity descends to visual maps related 
to eye movements (prefrontal eye field, parietal eye 
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FIGURE 8.33 Voluntary attention: From frontoparietal to 
sensory cortex. Voluntary attention in perception is directed to sen¬ 
sory cortex by frontal and parietal regions and perhaps also to other 
regions of the brain for motor, language, and executive functions. 
Notice the frontal lobe region, the frontal eye field, which exerts 
voluntary control over eye movements as well. Parietal regions 
are believed to be involved in spatially directed attention. Visual 
regions (in red) are enhanced by attentional mechanisms, such as 
gamma synchrony among multiple visuotopic maps for the selected 
spatial location and visual features. Source : Yantis et al., 2008. 


field, and superior colliculus) and visuotopical feature 
maps (Vl-IT). The pulvinar nucleus of the thalamus 
also contains a visuotopical map, and is hypothesized 
to bring together saliency cues, basically representing 
contrasting features and their locations in all the sen¬ 
sory feature maps. Notice that this brain model lacks a 
WTA mechanism, as postulated by the abstract model 
shown in Figure 8.32. 

Top-down attention is driven by expectations, and 
in the delay interval (see Figure 8.35A), subjects know 
where to look, but the stimulus has not yet appeared. 
Yet visuotopical synchrony still occurs in motion-sen¬ 
sitive areas like MT and the posterior intra-parietal 
sulcus (pIPS) on the right hemisphere. During this 
period of attentional expectancy there is significant 
coupling between MT and IPS. After the delay inter¬ 
val, the stimulus is presented to one side of the vis¬ 
ual field so that its first effect will occur in the brain 
region on the opposite side of the visual stimulus (the 
contralateral rather than ipsilateral field). In Figure 
8.35B, the stimulus presentation time, alpha, and high 
gamma synchronize regions of the right hemisphere. 





Voluntary control of 
attention 


visual objects represented in 
inferior temporal lobe 



FIGURE 8.34 A brain model for visual attention. Shipp (2005) explores a number of brain 
models of visual attention. The elements of this model are similar to Figure 8.31, but the compo¬ 
nents are attributed to specific brain regions. Notice that in addition to cortical maps, two subcorti¬ 
cal map-like regions are shown. They are the pulvinar nucleus of the thalamus and the superior 
colliculus. Many of these same regions are involved in the control of overt eye movements, raising 
the possibility that in evolution visuospatial attention may have emerged on the prior basis of selec¬ 
tive eye movements. Source: Adapted from Shipp, 2005. 
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FIGURE 8.35 Synchronizing multiple maps to enhance atten¬ 
tion. MEG in a spatially cued motion discrimination task shows 
high coupling via synchronized alpha, beta, low gamma, and high 
gamma among four regions of the (right) hemisphere. The right 
hemispheres have been mathematically 'puffed up' to make cortical 
areas hidden in sulci (folds) visible. By sending synchronizing sig¬ 
nals in different parts of the frequency spectrum, it is possible that 
the brain can send simultaneous signals that do not interfere with 
each other. Source : Siegel et al., 2008. 


During the delay period (see Figure 8.35B), anticipa¬ 
tory synchrony occurs prior to stimulus presentation 
but after subjects have been cued about the spatial 
location of the expected motion stimulus. In the stim¬ 
ulus interval (see Figure 8.35B), alpha plus low and 
high gamma can be seen. Notice that FEF, the frontal 
eye fields, which are involved in voluntary control of 
attention and eye movements, are decoupled during 
the stimulus interval. Thus, there is a rich pattern of 
coupling and decoupling between three important vis¬ 
ual attention regions, with four rhythmic bands being 
synchronized and desynchronized during various 
periods of the delay and stimulus periods. We could 
imagine the kinds of signals that might be exchanged 
between the synchronized brain regions as to enable 
flexible control of attention (See Figure 8.37). 


3*3 Attention versus conscious experiences 

Are 'attention' and 'consciousness' two labels for the 
same thing? That is an ongoing debate. We have taken 
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FIGURE 8.36 Separating conscious perception versus visual 
attention. Wyart and Tallon-Baudry (2008) presented an attentional 
task (like the flanker task described previously) in which a faint vis¬ 
ual grating was presented close to threshold. Eighty-five percent of 
trials contained a stimulus, about half above and half below thresh¬ 
old. Thus, it would be consciously detectable on about half the trials 
and undetectable (but still present) on the other half. On half of the 
stimulus-present trials, subjects were cued where to attend, and in 
the other half, they were not. Thus, attention was one independent 
variable (as defined by cued vs. noncued trials), and conscious¬ 
ness of the visual grating the other. Subjects were asked to identify 
the orientation of the visual grating and its presence or absence. 
Accuracy rates and reaction times were measured, and MEG was 
performed. Source : Wyart and Tallon-Baudry, 2008. 


the position that attentional selection often leads to 
conscious events and that conscious experiences can 
influence attention in return. That suggests that con¬ 
scious experiences and attentional selection can be 
separated but that they interact with each other. The 
traditional metaphor is to a spotlight in a darkened 
theater, in which the theater director can ask for the 
spotlight to shine on a specific actor or actress. The 
result of shining the spotlight is to bring out one of 
the people on stage. By analogy, the spotlight is some¬ 
thing that can select different players on stage, much as 
attention selects among a range of possible events. The 
conscious event corresponds to the appearance of one 
actor or another in the spotlight. In this view, attention 
and conscious contents are not identical to each other 
but are constantly interacting. 

A clever experiment by Wyart and Tallon-Baudry 
allowed for the conscious perception and selective 
attention to be manipulated separately in the same 
experiment (2008; Figures 8.36 and 8.37). 

It is not known at present whether the same areas 
in frontoparietal cortex are involved in both conscious¬ 
ness and attention, but there may be some overlap. 
When selected sensory information becomes conscious, 
it may activate some of the original frontoparietal loci 
that selected it in the first place, thus creating a self- 
sustaining feedback loop for some period of time. 
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FIGURE 8.37 Separate brain correlates of attention and awareness. Wyart and Tallon-Baudry (2008) 
were able to dissociate the effect of visual awareness and selective attention, (a) MEG recordings over the 
right occipital cortex from 100 to 500 ms post-stimulus, as shown in (b), the cartoon of the head. High lev¬ 
els of gamma power were observed over the occipital cortex. The lower graphs (c) show frequency times, 
displayed with colors indicating the F score of MEG power, a measure of statistical significance. The black, 
red, and yellow colors show high levels of gamma attributable to the awareness manipulation (left) and 
the cued attention manipulation (right). Visual awareness increased low gamma power at —250 to 500 ms 
(near 60 Hz), while attention involved a higher gamma burst around 80 Hz beginning 350 ms post-stimu¬ 
lus. This is somewhat counter-intuitive, since we might expect the attention effect to precede the awareness 
effect. One question is whether attentional effects could be detected earlier in frontoparietal regions of the 
brain. Notice that the significant gamma bursts were compared with the same locus in the time-frequency 
domain in each graph in post-hoc comparisons. Source : Wyart and Tallon-Baudry, 2008. 


4*0 REM DREAMS 

Not all typical dream experiences occur during REM 
states, as defined physiologically - by rapid, stereotypical 
eye movements, waking-type scalp EEG, and sensory- 
motor blocking. Quite a lot of subjective mental activity 
occurs during slow-wave sleep, although it does not tend 
to be the vivid narratives we experience during classical 
dreams. The sleep architecture of Figure 8.38 gives us a 
useful outline, but individual sleep stages may be quite 
different. We will therefore refer to 'REM dreams' as 


those periods when we have classical subjective, narra¬ 
tive dreams along with REM activity. 

Along similar lines, Hobson et al. (2000) define 
dreaming as 'mental activity occurring in sleep character¬ 
ized by vivid sensorimotor imagery that is experienced 
as waking reality, despite such distinctive cognitive fea¬ 
tures as impossibility or improbability of time, place, 
person and actions; emotions, especially fear, elation and 
anger predominate over sadness, shame and guilt and 
sometimes reach sufficient strength to cause awakening; 
memory for even very vivid dreams is evanescent and 
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FIGURE 8.38 The architecture of sleep. The thalamocortical core changes dramatically 
in sleep. In deep sleep, the low, irregular oscillations of waking EEG change to high, slow, and 
regular activity, called delta waves. The four levels of arousability (Stages 1-4) show increasing 
delta waves, until Stage 4 of Non-REM (NREM-4) is almost all delta. Every 90 minutes or so, 
sleepers show rapid eye movements (REM) as the eyes swing back and forth. At the same time 
the EEG shows striking waking-like activity in the EEG: fast, irregular, and low in amplitude. 
People who are awoken from REM tend to report vivid, narrative dreams, whereas SWS sleep¬ 
ers may report waking-like inner speech. REM periods become longer in the second half of the 
night. Source: Stickgold & Walker, 2007. 



tends to fade quickly upon awakening unless special 
steps are taken to retain it'. 

Both sensory input and motor output are blocked 
during REM dreaming, so that the brain is talking 
only to itself. The existence of motor inhibition may 
give rise to some 'paralysis dreams', in which we feel 
paralyzed, or want to move in the dream, but feel we 
cannot. This can lead to unpleasant Tocked-in' feel¬ 
ings during dreaming, if we become aware of being 
unable to move. But muscular inhibition is an entirely 
normal part of dreaming. For one thing, motor inhibi¬ 
tion keeps us from acting out our dreams. 

Memory is also impaired during REM dreaming. 
When dreamers are woken up from REM they may 
report vivid visual memories, but they tend to fade 
in seconds unless we make a special effort to rehearse 
them. It is almost as if we are creating a running movie 
in our dreams, but we can't recall the storyline from 
one scene to the next. Even people with good dream 
recall are likely to remember only a tiny fraction of 
their nightly 90 to 120 minutes of REM (Figure 8.38). 

REM dreaming is triggered by 'PGO waves' - 
pons-geniculate-occipital activation, along with neuro¬ 
modulation. Sharp bursts of neuronal spikes come from 
the pons (P), activate the lateral geniculate nucleus of 
the thalamus (G), and then trigger visual experiences by 
way of the occipital cortex (O). Hobson and McCarley, 
who advanced the early PGO hypothesis in 1976, called 
this the 'activation-synthesis hypothesis', with the 
emphasis on the activation burst from the pons and 
its conscious interpretation or 'synthesis' by the cortex 
(Hobson et al., 2000). An update is given in Hobson 


et al. (2000; Figure 8.39). Pace-Schott and Hobson (2002) 
write that REM sleep is also triggered by changes in 
major brain modulators, especially norepinephrine, 
acetylcholine, serotonin (5-HT), and glutamine. 

The PGO theory does not explain the content of 
dreams. It only suggests that the cortex tries to make 
a coherent story out of meaningless brainstem signals. 
What determines dream contents remains a puzzle. 
Traditional ideas advanced by Sigmund Freud and 
Karl Jung often are treated skeptically, but few plausi¬ 
ble alternatives have emerged. However, there is good 
agreement on the high emotional content of dreams. 

Revonsuo and colleagues have proposed a Threat 
Simulation theory of dreams, an evolutionary hypoth¬ 
esis based on the fact that the REM state is widely 
observed in mammals and birds, and therefore may 
have an important evolutionary function. Revonsuo 
(2000) suggests that dreams allow us to mentally 
rehearse threatening situations. Traumatized individ¬ 
uals may reexperience events related to the trauma (in 
some cases even during the waking state). Consistent 
with their theory, Valli and Revonsuo found that trau¬ 
matized children from a war zone had significantly 
more threatening dreams than controls. Other major 
life events, such as loss of a loved one, or even normal 
stressful events, might also be reflected in dream con¬ 
tents. Spontaneous waking thoughts tend to rehearse 
current concerns, and dream contents may repeat 
the same themes. It remains to be shown, however, 
whether dreaming also supports coping strategies, 
as we might expect if dreaming had an evolutionary 
threat-related function. 
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Lack of distinction between first- 
and third-person perspectives 


Lack of control on the 
flow of dream events 


Amnesia at awakening 


Bizarreness, discontinuity, and 
improbable combinations 


Uncritical acceptance of 
content 


Fragmented episodic 
memory recall 


Delusional belief 
of being awake 


Mind representation 


Emotions 


. 


Visual perception 


Motor perception 


FIGURE 8.39 Functional brain imaging of REM dreams. fMRI activity during dreaming. Red areas rep¬ 
resent increased blood-oxygen-related activity, for example in emotion areas. Blue areas correspond to below- 
baseline activity during dreaming, as in the frontal lobes. Higher activity in occipital regions may reflect vivid 
visual dream imagery. Low frontal lobe metabolism may also explain the delusional beliefs, bizarreness, and 
sudden discontinuities of dreams, such as rapid changes in scenes, dream characters, and narrative coherence. 
Low frontal lobe functioning is consistent with amnesia for dreams upon awakening, and low levels of volun¬ 
tary control. Source : Dang-Yu et al. in Squire, 2009. 


TABLE 8.3 Major features of REM dreams 


Hallucinations - especially visual and motoric, but occasionally in 
any and all sensory modalities 
Bizarreness - Incongruity (imagery is strange, unusual, or 
impossible; Discontinuity (imagery and plot can change, appear or 
disappear rapidly); Uncertainty (persons, places, and events often 
bizarrely uncertain by waking standards) 

Delusion - we are consistently duped into believing that we are 
awake (unless we cultivate lucidity) 

Self-reflection absent or greatly reduced - relative to waking 
Lack of orientational stability lack - persons, times, and places are 
fused, plastic, incongruous, and discontinuous 
Narrative story lines - explain and integrate all the dream elements 
in a confabulatory manner 

Emotions increased - intensified and predominated by fear-anxiety 
Instinctual programs - (especially fight-flight) often incorporated 
Volitional control - greatly attenuated 

Memory deficits - across dream-wake, wake-dream, and dream- 
dream transitions 


Source: Hobson, J.A in squire et al, 2008. 

4*1 Dreaming as a conscious state 

Dreams tend to be reported as conscious when people 
are woken up during REM and asked to describe what 


they were experiencing in the seconds before. The EEG 
of REM dreaming is strikingly similar to wakefulness, 
containing alpha, beta, gamma, and theta frequencies 
in a complex mix. It therefore seems that we actually 
have two daily conscious states: Waking and REM 
dreaming. The electrical activity of waking and dream¬ 
ing reflects a similar operating style by the thalamo¬ 
cortical system. However, it is important to keep in 
mind that there are important differences during REM 
as well, such as sensory and motor blocking, a high 
level of emotional activity, and a low level of executive 
(frontal lobe) control. 

The surreal and nonrational nature of dreams may 
reflect this lower level of executive control. Dreaming 
is a 'hypofrontal' state, like drowsiness, alcohol inebria¬ 
tion, delirium, and other states of lowered self-reflection 
and self-control (Dietrich, 2002). In addition, the limbic 
regions of the brain, which are involved with emotions, 
show higher fMRI activity than the waking state (Figure 
8.39 and Table 8.3). 

Some people learn to experience their dreams knowing 
they are dreaming, and even to exercise some volun¬ 
tary control during dreaming. LaBerge et al. (1981) 
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made a case for such 'lucid dreaming' in the sleep 
laboratory by showing that some dreamers could learn 
to voluntarily move their eyes back and forth three 
times in a row, on cue, when given an auditory signal. 
Some lucid dreamers could also count to ten in the 
dream, and then show the experimenters that they had 
done so by repeating the triple eye movement to sig¬ 
nal the end of the ten-second period. Lucid dreamers 
in this experiment had to recall and carry out their task 
instructions, learned during the waking period before 
going to sleep. Thus lucid dreaming required execu¬ 
tive functions and semantic memory retrieval, as well 
as linguistic and auditory processing. Lucid dreamers 
may therefore be able to control some of the cognitive 
functions that usually are impaired in the REM state. 

4*2 Memory consolidation in REM 

Debate continues over the role of REM dreams in mem¬ 
ory consolidation, with findings reported pro and con. 
However, emotional memories seem to be consolidated 
by REM dreams during afternoon naps - which are eas¬ 
ier to study than overnight dreams - when compared 
to emotionally neutral materials (Nishida et ah, 2009). 
Both emotions and REM involve ancient mammalian 
brain systems. There is a three-way overlap between 
brain regions for episodic learning (hippocampal com¬ 
plex and neocortex), emotion (hippocampus, amyg¬ 
dala, and neocortex), and dreaming (limbic activation, 
thalamus, and neocortex). Emotionally evocative expe¬ 
riences are the ones that humans need to remember if 
we are to improve our chances for reproductive suc¬ 
cess. We need to avoid those dangerous wild animals 
the next time we see them, or remember which of our 
clan members gave us food and shelter. Emotions have 
to do with our personal and interpersonal welfare: fear, 
attraction, safety, love, hunger and satiety, anger, envy, 
pride; those are the experiences we must remember. 

The stress hormone cortisol is involved in emo¬ 
tional experiences as well as memory consolidation. 
Cortisol increases during the night, as does the likeli¬ 
hood of REM dreaming (Payne and Lynn Nadel, 2004). 
More than 80% of SWS is concentrated in the first half 
of the night, whereas the amount of REM dreaming 
doubles in the second half. 

A number of studies have reported increased theta 
rhythm coupling between the hippocampus and fron¬ 
tal lobe during REM sleep. Since memory consolida¬ 
tion is believed to require that kind of theta-band 
coordination, those findings support a role for REM 
in the consolidation of memories (Hobson & Pace- 
Schott, 2002). However, the best emerging evidence 


for memory consolidation comes from advancing 
knowledge about gene expression required for protein 
manufacture needed to grow new synapses and other 
cellular structures needed for enduring memories. So 
far this is largely a literature on other mammals. The 
number of genes that are believed to play a role con¬ 
tinues to increase as we learn more (Figure 8.40). Thus 
we are in the midst of a major wave of new evidence, 
and we can cite only a few examples (see Chapter 16). 
Nevertheless, the evidence for consolidation through 
gene expression is likely to be more definitive than 
anything else we have so far. 

Figure 8.40 shows one kind of consolidation-related 
gene expression in the rat as a function of brain state. 
The gene is Zif-268, and its expression is fluorescence- 
tagged, so that immediately after learning, the rats are 
sacrificed and we can see color-coded fluorescent activ¬ 
ity in different parts of the brain. The figure shows six 
left coronal sections of the rat brain rapidly obtained 
during waking, slow-wave sleep, and REM. Notice 
that both the enriched environment and the control 
condition lead to moderate to high levels of Zif-268 
expression in the waking state (WK) (yellow and red 
colors). Most expression occurs during waking states, 
with REM playing a secondary role, and slow-wave 
sleep coming in last. Keep in mind, however, that these 
are still early days for the study of gene expression for 
memory consolidation. Hundreds of different genes 
are believed to be involved in learning, and this study 
is primarily an example of what is likely to emerge in 
the near future (Knapska & Kaczmarek, 2004). 


5.0 DEEP SLEEP: UPS AND DOWNS 

Slow-wave sleep (SWS) is a cognitively active state, 
even though it is the least conscious time of the daily 
cycle. SWS may allow the replay and consolidation of 
memories acquired during waking. 

One recent discovery is the role of 'slow oscilla¬ 
tions', defined as very widespread cortical waves as 
slow as 0.01 Hz, and peaking around 0.75Hz.There is 
debate whether Slow Oscillations (SO) are the lower 
end of the traditional delta range, which is easily seen 
in the raw EEG. Delta is considered to be 1 to 3 Hz, 
but all EEG frequency bands are somewhat tentative, 
because we continue to learn more about them. 

Slow Oscillations are seen as a series of gradual 
peaks and valleys in brain activity, known as Up and 
Down States. Each of those half-waves may last from 
0.5 sec to tens of seconds. The Up half-wave of the SO 
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FIGURE 8.40 Gene-controlled protein manufacture 
for memory consolidation. The best evidence for mem¬ 
ory consolidation comes from animal studies of gene 
expression needed to manufacture the proteins needed 
for memory consolidation. We are just beginning to see 
studies of perhaps hundreds of genes that are believed 
to be involved in memory consolidation. Above, slices 
of the left halves of rat cortex that were rapidly frozen 
immediately in each state, and subjected to fluorescent 
chemical labeling for the gene zif268. The primary com¬ 
parison is between the Enriched Environment, believed 
to support increased learning, and a Control condition. 
In the lower right hand slice, higher REM gene expres¬ 
sion is obtained in the Enriched Environment group 
compared to the Controls. Waking involves more Zif-268 
expression than REM, and REM more than slow-wave 
sleep. Source : Knapska & Kaczmarek, 2004. 


is believed to allow for a waking-like moment of brain 
activity (Destexhe et al., 2007). During the Up State 
neurons can freely signal to each other, so that theta 
and gamma interactions can take place. In contrast, the 
Down State is thought to lower thalamocortical activ¬ 
ity to its baseline level. The Up State of SWS therefore 
could be considered to be an active waking-like event, 
whereas the Down State seems to block most neuronal 
interactions. What is really distinctive about SWS there¬ 
fore might be the regular Down States, occurring every 
few seconds, and interrupting the waking-like Up 
States long enough to prevent integrative processing. If 
you imagine having to stop talking every other second 
in a normal conversation, you can see how much that 
would interfere with normal thinking and speaking. 

Slow Oscillations are believed to group faster wave¬ 
forms - theta, alpha, beta, and gamma. By analogy, 
the slow rise and fall of the ocean tides group six- 
foot beach waves that ride on top of the tides. Six- 
foot waves in turn group smaller choppy waves. In 
the actual sea, that kind of grouping by slower waves 
occurs by literally lifting up smaller waves, which flow 
on top of the slow tides and larger breakers. Such a 
hierarchy of waves makes for an attractive simplifica¬ 
tion, and as we will see, there is evidence to support it. 

A physiological mechanism for grouping of faster 
rhythms by slower ones is easy to imagine: The Up 
State of Slow Oscillations involves massive neuronal 


firing, making downstream neurons more likely to 
depolarize and fire their own axonal spikes. Neurons 
involved with faster waveforms therefore receive more 
dendritic inputs, and increase their graded potentials 
to come closer to the threshold of a depolarizing spike. 
Slow Up States can therefore be imagined as adding a 
voltage increment to large numbers of neurons, mak¬ 
ing them more likely to fire and interact. Down States 
would have the opposite effect. Such a very broad 
effect of slow waves might occur through volume con¬ 
duction, by means of widespread neuromodulation, or 
by neuron-to-neuron synaptic signaling (Chapter 3). 


5.1 Some mental activity occurs even in 
slow-wave sleep 

People awoken from slow-wave sleep report ongoing 
mental contents about half the time. Steriade (2006) 
writes that 'in contrast with the assumption that 
dreams occur entirely in REM sleep, a series of stud¬ 
ies, starting with the 1960s, has demonstrated the pres¬ 
ence of dream mentation during slow-wave sleep. 
Slow-wave sleep thinking is rational and repetitive, 
whereas during REM sleep the internal perceptions 
are vivid, thought becomes illogical, and the intensity 
of emotions is higher than during slow-wave sleep'. 
During REM sleep, dream recall may occur about 95% 
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Slow-Wave Steep 


FIGURE 8.41 Regional metabolism declines with depth of sleep. As the arousal threshold goes deeper, 
and sleepers are more difficult to arouse, more slow waves are visible in the EEG, and metabolic activity 
goes down in more parts of the brain. But deep sleep still has biological functions, as in the 'replaying' of 
hippocampal firing patterns of episodes acquired in the preceding day. Source : Adapted from Hobson, J.A in 
Squire, L. (ed.), 2009. 


of the time; outside of REM, reports of mental activity 
fall to about 50%. Steriade concludes that 'the brain is 
never 'empty', and mental activity is present during all 
stages of sleep' (2006, p. 1098). 

Why do we have the belief that deep sleep is com¬ 
pletely unconscious? One possibility is that arous- 
ability goes down as slow waves begin to dominate 
electrical brain activity (Figure 8.18). Lower arousabil- 
ity means that it takes longer to wake up, and a longer 
time to full arousal means that our memory for sleep 
mentation may be worse. Memory can fade in seconds. 
Some individuals 'wake up very slowly', suggest¬ 
ing that their frontal lobe function does not reach full 
waking levels for some time after they become per¬ 
ceptually conscious. In the study of the circadian sleep 
cycles some people are called 'larks' (they feel alert in 
the morning) and 'owls', who feel groggy in the morn¬ 
ing and have their peak of alertness later in the day. 

5*2 The arousal threshold changes during sleep 

The depth of sleep is measured by the arousal 
threshold - whether a human or animal will become 
alert to a stimulus and orient to it. The arousal threshold 
goes up as slow waves increase in the EEG (Figure 8.41). 
Most slow-wave sleep tends to occur earlier in the night, 
and most REM dreams happen in the second half. 


Figure 8.41 shows lower metabolic activity spread¬ 
ing in the brain as sleep becomes deeper (as defined by 
ease of arousal). Brain metabolism declines in a specific 
order, beginning with midline cingulate structures that 
have been associated with 'self-orientation' and their 
corresponding thalamic nuclei. In slow-wave sleep 
(Stages 3 and 4) metabolism is decreased in the active 
'cognitive' parts of the brain. However, sensory cortex 
remains relatively active, although the flow of sensory 
stimuli is blocked at thalamic relay nuclei. 

5*3 Memory replay and consolidation 

It is difficult to study the effects of SWS separately from 
REM, because they constantly go together. However, SWS 
is more common in the first half of the night, whereas 
REM predominates in the second half. Researchers 
therefore have studied memory consolidation after 
'SWS-rich' and 'REM-rich' periods of sleep. In general, 
SWS-rich periods appear to strengthen explicit memo¬ 
ries, and REM-rich periods strengthen procedural tasks 
and perhaps implicit memories. Increased SWS has been 
observed after intensive episodic learning, and increased 
REM after procedural training. 

Other studies focus on the role of the hippocampal 
region in the acquisition of episodic (conscious) mem¬ 
ories (Figures 8.42 and 8.43). Episodic memories are 
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FIGURE 8.42 Slow-wave sleep enables memory consolidation during "Up States'. How slow-wave 
sleep is believed to enable memory consolidation. Three brain regions work together: Early episodic 
learning during the waking state encodes labile (unstable) hippocampal and neocortical memory traces. 
During slow-wave sleep thalamic spindles ignite Up States, which triggers hippocampal sharp spikes, 
which finally are believed to activate synchronized memory traces in the hippocampus. Thus thalamic 
spindle activity helps to trigger the next Up State, as shown by cortical iEEG. These regions act in concert 
to reactivate recent memory traces. During the following Down State of SWS the activated neurons are 
believed to express proteins that lead to synaptic plasticity for more enduring memory coding. Adapted 
from Steriade, 2000. 


thought to be transferred from the hippocampus to the 
neocortex in order for memories to become enduring. 
In rodents the hippocampal region has been studied for 
decades, using direct brain recording in maze learning. 
The hippocampus plays a major role in place learning 
and navigation, and it has been possible to pinpoint 
hippocampal 'place cells' that fire when the animal 
passes a particular point in the maze. As the rat runs the 
learned maze, place cells fire in sequence. Afterward 
the same pattern of cell firing is observed during slow- 
wave sleep, activating other parts of the brain, like the 
thalamus, neocortex, and basal ganglia (Figures 8.42 
and 8.43). In sum, in rats we can actually record neurons 
when the animals are exploring locations in sequence, 
and observe that the brain replays a learned spatial 
sequence during SWS. The theta rhythm appears to be a 
major organizing and transmission rhythm for the hip- 
pocampal-neocortical dialogue that leads to memory 
consolidation (Jensen, 2005; Buzsaki, 2002). 



FIGURE 8.43 Hippocampal Sharp and Ripple Waves. A 
cartoon of the hippocampus shows its basic arrays of neurons, 
which generate Sharp Waves and Ripple Complexes in activating 
retrieval and neocortical consolidation during slow-wave sleep. 
Source : Beenhakker and Huguenard, 2009. 


Humans have also been studied in learning to navi¬ 
gate a virtual town, using a video game. After learn¬ 
ing the town, similar hippocampal activity was again 
found to reactivate during SWS. The amount of reac¬ 
tivation correlated with recall on the following day. 
Thus the human and animal evidence points to the 
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TRENDS in Cognitive Sciences 

FIGURE 8.44 Artificial induction of slow waves. Individual 
slow oscillations can be enhanced by electrical field induction over 
the scalp. Notice that induced slow oscillations also evoke thalamic 
spindles and hippocampal sharp waves, which work together 
to enable hippocampal memory retrieval and consolidation. 
Source : Marshall and Born, 2007. 


same basic conclusions (Colgin et al., 2009; Vertes & 
Kocsis, 1997). 

The hippocampus has many of the functions of the 
brain as a whole, because it was once the core brain of 
ancestral mammals. The larger hippocampal system is 
involved in spatial navigation and in marking emotion¬ 
ally significant experiences, as well as in goal-guided 
control of behavior, episodic and declarative memory 
encoding and retrieval, and in SWS memory con¬ 
solidation. Different neuronal arrays (CA1 and CA3) 
generate Sharp Waves and Ripples, which are multi¬ 
plexed together (as in the Sharp Wave-Ripple Complex 
on the right). As the previous figure shows, these 
rhythms enable retrieval of active memories learned 
during the waking state, and their replay and consoli¬ 
dation in the neocortex. 

Some of the best evidence for SWS memory consoli¬ 
dation comes from the use of transcranial brain stimu¬ 
lation to entrain slow oscillations in the cortex. Once 
these oscillations are under experimental control, direct 


causal hypotheses can be tested. Figure 8.44 shows 
how it is done. A subject learns a task and goes to sleep 
several hours later in a sleep laboratory, wearing an 
EEG electrode cap. As natural SWS is observed in the 
EEG, an electromagnetic coil located over the head is 
pulsed slowly (<lHz), to enhance the slow oscilla¬ 
tions. Simultaneous recording shows a slow wave in 
the cortex, along with typical thalamic spindle waves 
and hippocampal sharp waves (SPWs). 

In a sham control condition the identical procedure 
is performed without electromagnetic stimulation, to 
control for placebo effects. Transcranial slow-wave 
stimulation during SWS was found to improve epi¬ 
sodic but not procedural memories. It seems that arti¬ 
ficial slow-wave induction improves the natural SWS 
conditions for memory consolidation. 

Notice that we now know several brain rhythms 
that play essential roles in memory consolidation dur¬ 
ing SWS. 

1 Slow oscillations cycle at < 1 Hz, but may overlap 
with the delta range of 1-3 Hz. During Up States, 
the peaks of slow oscillations, fast neuronal firing, 
and interactions are possible, much like the wak¬ 
ing state. But during the Down States of SWS, the 
cortex and thalamus go silent. 

2 The slow Up State also groups theta activity, 
cycling at 4-7Hz. Theta is known to be involved 
with hippocampal organization of learned infor¬ 
mation, and with transmission of that information 
to the neocortex. 

3 Theta waves 'group' faster gamma activity, both 
within the hippocampal complex, and in encod¬ 
ing sensory information in the hippocampus 
(from visual cortex, for example). Theta also car¬ 
ries hippocampal messages to the frontal lobes 
and other parts of the neocortex during memory 
consolidation. 

4 Gamma activity (30 Hz and above) is embedded in 
theta activity, and also serves to synchronize hip¬ 
pocampal and neocortical neurons. A number of sci¬ 
entists believe that it is gamma activity that actually 
carries the most detailed information, grouped by 
slower oscillations. Gamma is now known to have 
a very wide range (by some estimates, 25-120 Hz, 
with some activity even at 600Hz and 1200 Hz). 

In all these cases, slower waves are believed to increase 
the probability of the next faster wave. The Up State of 
the Slow Wave enhances theta, theta enhances gamma, 
and gamma recruits synchronous firing of individual 
neurons and groups of neurons in the hippocampus 
and neocortex. 
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6*0 PUTTING IT ALL TOGETHER 

Consciousness has intrigued humans since the begin¬ 
ning of history. With improved brain and cognitive 
techniques we can see considerable progress in recent 
years. We will review this chapter by looking at some 
popular proposals about consciousness and attention. 

6♦ 1 Does consciousness reflect a global 
workspace function in the brain? 

The idea that consciousness has an integrative function 
has a long history. Global Workspace Theory (GWT) is 
a cognitive architecture with an explicit role for con¬ 
sciousness in humans. It suggests that conscious con¬ 
tents involve a fleeting integrative memory capacity 
in the brain, in which only one consistent content can 
be dominant at any given moment (Baars, 1988, 2002). 
Potentially conscious events compete for access to the 
limited-capacity workspace. In the attentional model 
described earlier, this competition is accomplished by 
a Winner-Take-All network (Itti & Koch, 2000). 

GWT proposes that momentarily dominant infor¬ 
mation is widely distributed in the brain. That makes 
sense in a nervous system viewed as a massive distrib¬ 
uted set of special-purpose networks. In such a system, 
coordination, control, and novel problem solving could 
take place by way of a central information exchange, 
allowing some regions - such as sensory cortex - to dis¬ 
tribute information to the whole. This strategy is par¬ 
ticularly useful for dealing with novel problems that do 
not have a known solution, and that may be solved by 
the collaborative and/or competitive activation among 
numerous specialized networks, each of which may 
contribute toward a solution. Such an approach works 
well in large-scale computer architectures, which show 
typical limited capacity behavior when information 
flows by way of a global workspace (Baars, 2002). 

A body of evidence now indicates that conscious¬ 
ness is the primary agent of such a global access func¬ 
tion in humans and other mammals (Baars, 2002; 
Dehaene, 2001). Dehaene and colleagues used visual 
backward masking to compare conscious and uncon¬ 
scious words, using a pattern mask presented imme¬ 
diately afterward. Masked words are unconscious, but 
they are not physically blocked from entering the eyes. 
Brain imaging analysis showed that both conscious 
and unconscious words activated the cortical vision 
and word recognition areas, which are known to ana¬ 
lyze stimulus location, shape, color, and word identity. 
However, conscious words triggered 12 times more 


activity in these regions than identical unconscious 
ones. In addition, conscious words evoked a great deal 
of additional activity in the parietal and frontal cor¬ 
tex. It seems as if the stimulus-related activity in the 
conscious case may be widely distributed from visual 
regions to other areas in the brain. 

Conscious involvement is particularly useful when 
novel information needs to be combined and integrated, 
and when it needs to recruit additional brain resources. 
That is presumably why episodic memory (memory for 
conscious events) is so central in brain anatomy, with 
the hippocampal complex and neocortex engaged in a 
constant dialogue for storing and retrieving episodic/ 
declarative memories. Even implicit learning is mobi¬ 
lized by conscious cues, as described in this chapter. 

GWT generates testable hypotheses regarding con¬ 
scious and unconscious aspects of perception, learning, 
working memory, voluntary control, and self systems 
in the brain. It has close similarities to other biological 
theories such as Neural Darwinism and dynamical the¬ 
ories of brain functioning (Edelman & Tononi, 2000). 
Dehaene and others have developed experimentally 
testable models for aspects of the theory (Dehaene, 
2001). A number of experiments have led to generally 
positive results. 

Dehaene and colleagues (2001) proposed a neuro¬ 
nal implementation of a global workspace architecture, 
called neuronal global workspace (Figure 8.45). In this 
model, sensory stimuli mobilize excitatory neurons with 
long-range cortico-cortical axons, leading to the genesis 
of a global activity pattern among workspace neurons. 
Any such global pattern can inhibit alternative activity 
patterns among workspace neurons, thus preventing 
the conscious processing of alternative stimuli. The glo¬ 
bal neuronal workspace model predicts that conscious 
presence is a nonlinear function of stimulus salience 
(i.e., a gradual increase in stimulus visibility should be 
accompanied by a sudden transition of the neuronal 
workspace into a corresponding activity pattern). Such 
'ignition' of the workspace is consistent with the notion 
that consciousness is an all-or-nothing, as opposed to 
a graded, process. The neuronal global workspace has 
been used to model a number of experimental phenom¬ 
ena in consciousness. In one study, spontaneous work¬ 
space activity blocked external sensory processing, in a 
manner analogous to inattentional blindness, in which 
normal subjects engaged in effortful mental activity fail 
to notice salient but irrelevant stimuli. 

A similar phenomenon is the attentional blink, in 
which subjects are unable to report a stimulus S2 pre¬ 
sented within a short time interval (100-500 ms) follow¬ 
ing the presentation of a first stimulus SI. In a neuronal 
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FIGURE 8.45 A neural global work¬ 
space to support conscious experiences. 
Dehaene et al. (2001) proposed a neuronal 
global workspace model, attributed to long¬ 
distance interconnectivity of the prefrontal 
cortex. Other specialized regions of the cortex 
are selectively mobilized into the conscious 
workspace. Most brain networks are acti¬ 
vated automatically (unconsciously). The 
model accounts for major brain functions 
like perceptual categorization and long¬ 
term memory, as well as experimental para¬ 
digms like the attentional blink and visual 
backward masking. Source: Dehaene, with 
permission. 


global workspace model of this paradigm, the pres¬ 
entation of SI evoked both bottom-up and top-down 
signal flow, leading to the sort of sustained activity 
that might underlie conscious reportability in human 
subjects. However, the presentation of S2 immediately 
after SI evoked only bottom-up activity because of the 
global activity constraints imposed by SI. This decay of 
sustained activity relating to S2 may reflect the lack of 
conscious access in the attentional blink effect. 

A number of scientists have now published evi¬ 
dence consistent with this framework. The broad¬ 
casting hypothesis is probably the most distinctive 
hypothesis. That is, conscious contents are proposed to 
recruit multiple brain resources, a process that involves 
global distribution of information (both in the brain, 
and in the original artificial intelligence models that 
utilized a global workspace). In a 'society model' of the 
brain, comparable to the World Wide Web, the details 
of information processing are left to a large number 
of local networks. The role of the global workspace, 
like a chat room on the Web, is to allow the interaction 
of multiple streams of goal-directed processes, and 
to broadcast dominant messages from among those 
potentially competing streams. As in an Internet chat 
room, some voices may be more dominant than oth¬ 
ers, and the most consistently dominant ones have the 
widest outreach. At the same time multiple processing 
streams are running outside of the chat room, or in this 
case, outside of the global workspace. 


A number of findings show that conscious cognition 
is distinctively associated with the wide spread of cor¬ 
tical activity, notably toward frontoparietal and medial 
temporal regions (see Chapter 1). It is essentially to 
use unconscious comparison conditions in experimen¬ 
tal tests of any theory of conscious functions. Current 
evidence indicates that unconscious comparison condi¬ 
tions tend to activate only local regions, such as local 
visual processes. These findings are consistent with 
GWT, which is now favored by a number of scientists 
and philosophers (Baars, 2002). Baars and Franklin 
(2005) also have proposed that conscious moments are 
essential to recruit Working Memory functions like men¬ 
tal rehearsal, encoding, retrieval, and voluntary report. 

An alternative hypothesis is that, because any con¬ 
scious event is presumed to be reportable, the prefrontal- 
parietal component may be related to readiness to 
report rather than conscious experience as such. This is 
a difficult alternative to test conclusively. As we gain a 
more detailed understanding of the specific functional 
regions of the cortex, it may become possible to do so. 

Many sources of evidence suggest that conscious 
events show wider cortical activity than matched com¬ 
parison cases. A major theoretical question, therefore, 
concerns the contribution of frontoparietal activity in 
conscious perception. 

Unconscious states show the other side of the glo¬ 
bal broadcasting hypothesis: they show frontopari¬ 
etal hypometabolism in deep sleep, coma, vegetative 
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FIGURE 8.46 Long-distance gamma phase-locking in Waking, SWS, and REM. The upper row shows iEEG neocortex and 
the lower row shows hippocampal-cortical iEEG. Waking shows consistent gamma phase-locking averaging near 50%. Gamma is 
defined as 35-58 Hz. The horizontal axis refers to percent of all possible combinations of electrodes showing phase locking. (Similar 
values were obtained for state-dependent local gamma coherence.) Source : Cantero et al., 2005. 


states, epileptic loss of consciousness, and general 
anesthesia (see Baars et ah, 2003; Blumenfeld, 2008). We 
are seeing broadly consistent lines of evidence. 

Figure 8.46 from Cantero et ah (2005) shows that 
long-distance gamma phase-locking, which reflects fast 
information transmission, is distinctively associated 
with the waking state, but not SWS or REM. This study 
was performed in presurgical epilepsy patients with 
implanted intracranial electrode grids, and therefore 
represents the highest signal-to-noise ratio obtainable 
at the present time. Notice that long-distance gamma 
phase locking occurs both within the neocortex itself, 
and also between the hippocampus and cortex. 

As mentioned before, gamma activity is believed to 
be carried on slower wave forms. The 100-ms domain is 
especially important, since it represents the minimum 
conscious integration time in perception, for example. 
That is, two discrete sensory events in any modality 
can be integrated into a single conscious event only if 


they occur within 100 ms of each other (approximately). 
The 100-ms integration time does grow much larger 
in speech or music perception, or dance performance, 
where momentary events are interpreted in a much 
longer contextual framework. But for separate conscious 
events, like fast clicks, brief tones, visual flashes, sensory 
taps, and so on, the approximately 100-ms integration 
time seems to hold up well. Other sources have also sug¬ 
gested a 100 ms or 10 Hz series of conscious microstates, 
due to evidence from EEG and other sources (Freeman, 
2006; Lehmann, 2010). 

It is therefore interesting that Doesburg and col¬ 
leagues (2008), using MEG recording of visual stim¬ 
uli, found approximately 100 ms momentary gamma 
phase-locking across large regions of cortex in response 
to lateralized visual signals. This is precisely what we 
might expect for a momentary conscious 'broadcast' of 
momentary visual inputs (Figure 8.47). Notice that there 
is no synchrony in the first 200 ms after the stimulus. 
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FIGURE 8.47 A moment of conscious activation? Doesburg et al. (2007) used MEG to 
observe long-range gamma synchrony bursts 300 ms after the onset of a visual cue lateralized 
to the right or left side. Significant synchrony at 39 Hz between sensor locations is indicated by 
black lines, and desynchrony is shown by white lines. A visual stimulus was presented to the 
right or left of the fixation point. At 100 and 200 ms, desynchrony predominated, with the emer¬ 
gence of some long-distance synchronized oscillations (black lines). At 300-400 ms synchrony 
dominated at numerous sensor sites. In the right-most column only parietal sensors are shown. 
Black lines indicate high synchrony contralateral to the visual stimuli, as would be expected from 
the cross-wired anatomy of the cortex. From parietal sources (P7 on the left, and P8 on the right) 
activity can be seen to spread to both ipsilateral and contralateral regions of cortex. This evidence 
is consistent with the hypothesis of global brain-wide broadcasting during a conscious moment 
of about 100-300 ms in duration. Since theta and alpha cycle at about that rate, the claim is that 
alpha or theta may carry gamma phase-locked activity in multiples of 100 ms bursts. Source : 
Doesburg et al, 2007. 


but that from 300 to 400 ms a brainwide burst of 
gramma phase locking occurs. (Phase-locking involves 
synchrony with a possible brief lag time, and is used 
here as a more inclusive term.) The right-most column 
of Figure 8.47 shows only the parietal sensors on the 
other side of the cortex from the visual signal (i.e., con¬ 
tralateral) making momentary gamma phase-locking 
bursts for about 100 ms, virtually a visual picture of 
what is meant by 'global broadcasting' in the brain. 
These data are consistent with other studies showing a 
special role for theta or alpha signals, since these slower 
waveforms oscillate near 10 Hz, and therefore might 
provide the appropriate carrier waves for gamma 
phase-locked fast oscillations. 

Notice that we now have a number of separate 
sources of brain evidence showing coherent gamma 
bursts in association with conscious perceptual 
moments, paced by rhythms near theta or alpha. 


In order to test whether this applies to conscious, but 
not necessarily unconscious, events, a number of labo¬ 
ratories have used binocular rivalry, visual backward 
masking, the attentional blink, and other methods. 
Doesburg et al. (2009) used binocular rivalry to 'Reveal 
Large-Scale Oscillatory Network Dynamics Mediating 
Visual Perception'. They write. 

We propose that discrete moments of perceptual expe¬ 
rience are implemented by transient gamma-band 
synchronization of relevant cortical regions, and that 
disintegration and reintegration of these assemblies is 
time-locked to ongoing theta oscillations. 

Using somewhat different experimental methods and 
analytical tools, Gaillard et al. (2009) also concluded 
that 

... nonconscious processing of masked words was 
observed in multiple cortical areas ... but without 
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Unmasked Effect 
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FIGURE 8.48 Long-distance gamma synchrony in conscious, 
but not unconscious vision. The difference between conscious and 
unconscious visual input is shown in this experiment by a brief but 
statistically significant long-distance gamma synchronous burst 
slightly above 20 Hz. (Arrow) Four convergent measures signaled a 
difference between the two conditions, including Granger causality, 
which allows the statistical extraction of causally related activities 
in complex and parallel time series, such as we constantly see in the 
brain. Source : Gaillard et al. r 2009. 


coherent long-distance neural activity, suggesting a 
quickly dissipating feedforward wave. In contrast, con¬ 
scious processing of unmasked (conscious) words was 
characterized by the convergence of four distinct neu¬ 
rophysiological markers: sustained voltage changes, 
particularly in prefrontal cortex, large increases in 
spectral power in the gamma band, increases in long¬ 
distance phase synchrony in the beta range, and 
increases in long-range Granger causality. We argue 
that all of those measures provide distinct windows 
into the same distributed state of conscious processing. 

Figure 8.48 shows the critical difference between con¬ 
scious and unconscious stimulus conditions in the 
experiments by Gaillard et al. It appears as a single, 
high-amplitude synchrony burst just above 20 Hz, and 
400 to 550 ms after stimulus onset. This graph repre¬ 
sents a subtraction between the conscious and the 
unconscious conditions (called the 'Unmasked Effect' 
because the control was a visually backward-masked 
and therefore unconscious stimulus). 

The convergence between different sources of evi¬ 
dence is by now striking. When careful brain monitor¬ 
ing is performed, especially using sensitive measures 
like intracranial recording, brief gamma phase-locking 
between distant parts of the cortex is observed in con¬ 
scious, but not closely matched unconscious condi¬ 
tions. This gamma phase-locking appears to be carried 
by somewhat slower waveforms in the theta-alpha 
frequency domain. We have seen similar effects in a 


variety of studies throughout this chapter. At a gross 
level the same kinds of activities can be recorded in 
the surface EEG and MEG, in changes of conscious 
state during sleep and other unconscious conditions, 
and in a number of cognitive tasks. Such a conver¬ 
gence between different sources gives us greater con¬ 
fidence that the emerging understanding of these hotly 
debated questions is indeed real. 


6.2 Reentrant signaling and complexity in 
conscious brain events 

Because the thalamocortical complex has interactive 
connectivity compared to other structures like the cer¬ 
ebellum, this part of the brain has been the focus of this 
chapter. Gerald Edelman and colleagues have provided 
detailed theoretical studies of the thalamocortical sys¬ 
tem, in the framework of Neural Darwinism (ND). ND 
suggests that brain development and dynamics are selec¬ 
tionist in nature, not instructionist, in contrast to comput¬ 
ers, which carry out explicit symbolic instructions. 

Selectionist processes have four basic features, anal¬ 
ogous to living species: 

1 A set of elements (neurons, synapses, or dynamic 
cell assemblies) is characterized by diversity, such 
as populations of neuronal groups in the brain. 

2 These elements can reproduce or amplify. 

3 A process of selection operates on the products of 
diversity; for example, differential reproductive 
success serves to select some offspring and not 
others. 

4 'Degeneracy' or functional redundancy, the ability 
of different combinations of elements to perform 
the same function, is inherent in such systems. 
Functional redundancy is a standard feature of 
human engineered systems, just as it is a routine 
feature of biological organisms. 

In the brain, selectionism applies both to neural 
development and to moment-to-moment function¬ 
ing. Selection in the brain is directed by value, which 
reflects the salience of an event and can be positive 
or negative. Value is similar to selective pressure in 
evolution. In the brain, it involves pleasure, pain, and 
emotional salience. Finally, according to ND, spa- 
tiotemporal coordination in the brain is enabled by 
reentrant signaling, the recursive exchange of signals 
among neural areas across massively parallel recipro¬ 
cal connections. 

According to Gerald Edelman, the neural systems 
underlying consciousness arose in evolution as a result 
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of their ability to integrate a large number of sensory 
inputs and motor outputs in parallel. This allowed for 
the discrimination of sensorimotor signals in a high¬ 
dimensional space, allowing for the selection of adaptive 
behavior. Primary (sensory) consciousness is associated 
with an interaction between perceptual categorization 
and memory, where primary consciousness refers to the 
presence of a multimodal scene of perceptual and motor 
events. It provides an animal with discriminatory selec¬ 
tivity, flexibility, and planning capacity. 

Edelman, along with other theorists, distinguished 
between 'primary' consciousness and 'higher-order' 
consciousness which in humans is associated with lan¬ 
guage, the temporally stable self, autobiographical mem¬ 
ory, and the ability to imagine and plan for the future. 

In the dynamic core hypothesis Edelman and col¬ 
leagues (2000) propose that any particular conscious 
event combines a high level of information-rich dis¬ 
crimination and integration. The neural mechanisms 
of a conscious experience involves a functional cluster 
in the thalamocortical system, within which reentrant 
neuronal interactions yield a succession of unitary 
states. The boundaries of a dynamic core may change 
over time, with some neuronal groups leaving and 
others being incorporated. 

According to Tononi and Edelman (2000), high val¬ 
ues of a measure called 'neural complexity' (C) charac¬ 
terize conscious events. C reflects the extent to which 
the dynamics of a neural system are both integrated 
and differentiated. Modeling studies suggest that the 
thalamocortical system is suited to producing dynam¬ 
ics of high neural complexity. 

Tononi has suggested that consciousness corresponds 
to the capacity of a neural system to integrate informa¬ 
tion, defined formally as combining a large repertoire 
of states, such as a visual experience of an object like a 


computer screen. This information-theoretical proposal 
is based on the notion that the occurrence of any con¬ 
scious scene simultaneously rules out the occurrence of 
a vast number of alternatives and therefore constitutes a 
highly informative discrimination. Tononi has proposed 
a formal measure of the degree of consciousness that is 
enabled in a system. This measure, called phi, is defined 
as the amount of causally effective information that 
can be integrated across the weakest link of a system. 
According to the theory, consciousness as measured 
by phi is characterized as a disposition or potentiality. 
Both neural complexity and phi can be considered to 
measure the spatial aspect of the balance between dif¬ 
ferentiation and integration in a neural system. 

Seth and colleagues (2006) have suggested useful 
additional measures of neural dynamics to measure 
other aspects of this balance, including complexity 
across time (temporal complexity) and across different 
levels of description (recursive complexity). Temporal 
complexity reflects the fact that consciousness extends 
over time in several ways. Many experiments sug¬ 
gest that it takes 100 ms for sensor stimuli to be incor¬ 
porated into a conscious scene and that neuronal 
readiness potentials can appear several hundred mil¬ 
liseconds before the awareness of an intention to act. 
Recursive complexity reflects the fact that brains show 
rich organization at multiple levels, from molecular 
interactions within individual synapses to reentrant 
interactions among distinct brain regions. A combina¬ 
tion of measures may be needed to adequately charac¬ 
terize the neural dynamics relevant to consciousness. 

Edelman, Tononi, and Sporns have proposed that out 
of four potential connectivity patterns in the thalamocor¬ 
tical system (Figure 8.49) the one that enables conscious 
functions is the formal pattern shown on the right, called 
'Complexity'. A purely random connectivity machine 
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FIGURE 8.49 Four possible thalamocortical patterns of connectivity. In recent years we have seen 
progress in understanding the connectivity pattern of the core brain, the thalamocortical system. Some pro¬ 
posals about the conscious state and its contents make claims about the kind of dynamic connectivity that 
may be needed for a conscious brain. For example, they aim to answer why the cerebellum is not considered 
to support conscious contents, while the cerebral cortex and thalamus, in combination, are believed to do that. 
Source : Sporns and Tononi, 2002. 
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is unable to sustain specific contents; an entropy sys¬ 
tem loses information. The third connectivity pattern, 
integration, is very useful for specific biological func¬ 
tions, but fails to have the high level of flexibility that 
characterized conscious brain functions. Complexity is 
a formal measure that allows for high-dimensional dis- 
criminability between different inputs and outputs, and 
is also able to integrate very high numbers of such states 
in a unitary fashion. These proposals can be formalized, 
and have been tested against detailed models of the 
thalamacortical system, the cerebellum, and other parts 
of the brain (Izhikevich & Edelman, 2008). 


63 Does consciousness require an 
experiencing self? 

Intuitively most of us believe that conscious experi¬ 
ences involve an 'experiencing self' - a personal view¬ 
point on the world that may be supported by parietal 
and prefrontal areas of the cortex (Baars et al., 2005; 
see Chapter 12). In daily life we use phrases like T'm 
awake', 'I see the coffee cup', 'I lost consciousness', 'I 
couldn't control myself', and so on. However, there is 
a large philosophical literature that is critical of this 
commonsense idea. For many years the 'observing I' 
was ridiculed as 'the homunculus fallacy' (from the 
Latin word homunculus , meaning Tittle man'). The 
problem, according to the critics, is that to make sense 
of the observing self, a little observer would have to 
sit in the brain, looking at the sensory inflow. But to 


make sense of the little observer inside the brain, it 
would also need a little observer inside, and so on ad- 
inf initum. Such an infinite regress does not explain 
anything. 

However, not all versions of an observing self lead 
to an infinite regression (Baars, 1988). It is possible to 
have an executive capability in the frontal and parietal 
lobes, one that is able to interpret information from the 
senses in terms that are important for decision-mak¬ 
ing - like: 'Is this news good for me? If not, will it hurt? 
How much? Should I run away? Or would I feel embar¬ 
rassed if I did'? There is a great deal of evidence for an 
executive network in the human brain (Chapter 12). 

The weight of scientific opinion may now be swing¬ 
ing back to the idea of an executive T. The brain 
regions in the frontal lobe shown in Figure 8.50 are the 
same that were damaged in the classic case of Phineas 
Gage, and other frontal lobe patients who suffer from a 
radical change in personality and impaired self-control. 
The sense of agency in voluntary control is also 
dependent on the medial and lateral frontal regions, 
and the right parietal region (above) is a crucial area 
for 'perspective of the self' in the neurological condi¬ 
tion of parietal neglect (Baars, 2002; Baars et al., 2003). 

Is an 'I system' needed for conscious experiences? 
For that reason, this emerging literature may give us 
the first scientific handhold on these notoriously dif¬ 
ficult questions. 

The right PPC may be one of the brain areas that 
support 'the perspective of the self' and the sense of 
personal agency in action control. The posterior parietal 
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FIGURE 8.50 The precuneus /medial pre¬ 
frontal region may support the "self as observer'. 
Self-related regions of the brain are also high-traffic 
hubs, as indicated by mapping studies of the white 
matter - the major signaling hubs and highways 
of the cortex and thalamus. Brain connectivity fol¬ 
lows 'small world' mathematics, both structurally 
and functionally, as measured by 'signal traffic 
flow'. Source : Hagmann et al, 2008. 
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cortex may also be involved in loss of consciousness 
due to anesthesia (Alkire & Tononi, 2008). 

William James pointed out that the 'I as observer' is 
not the same as the 'me' as an object of perception and 
thought. That is, the T is not the face you see in the 
mirror, nor is it the objectified concept of ourselves as 
others may see us. Rather, the T is the point of view of 
the perceiver, the volitional actor and thinker. Of course 
it is quite possible that the T is an inferred entity, and 
even one that is mythical, or culturally acquired. After 
all, human beings constantly make inferences about all 
kinds of nonexistent entities in the world, and the mere 
fact that we assert the existence of something does not 
prove that it does exist. However, the perspective of 
the T seems to be universal across cultures and fun¬ 
damental. Young children seem to have a sensorimo¬ 
tor perspective from very early on, and presumably an 
emotional one as well. Many of our emotions have to 
do with experiences of well-being and danger to the 
experiencing self. To feel hurt is equivalent to a nega¬ 
tive evaluation of the experiencing self. Other emo¬ 
tions, like shame, guilt, and pride, are thought to be 
'self-conscious emotions'; that is, feelings that reflect 
how we believe we are perceived in the social world. 


6A Why are conscious events reportable? 

Our basic evidence for the fact that others are con¬ 
scious is what they tell us - which fortunately coin¬ 
cides with our own experience much of the time, at 
least in the perceptual world. But any scientific theory 
must ultimately also explain its own most basic evi¬ 
dence. So far, there are not many attempts to explain 
why it is that conscious events are reportable. 

A somewhat trivial explanation is that the forward 
'global broadcasting' of perceptual contents from sen¬ 
sory cortices to the frontoparietal regions also activates 
language areas, like Broca's area, which are needed 
to report conscious events. But that is not really good 
enough. People who are aphasic can still report con¬ 
scious experiences, and even people who are para¬ 
lyzed can make use of eye movements to fixate letters 
on a large keyboard display, to Type' their conversa¬ 
tions. The output modality of speech is not a necessary 
requirement for being conscious about events. 

At a somewhat deeper level we may argue that per¬ 
ceptual objects and events can be pointed out, a skill that 
even very young children can use. Children and their 
caretakers constantly play the game of 'Look! There's 
a toy'! or later, 'Look! There's a (fill in the name)'. It is 
obviously an important part of our growing up to know 


the shared social and interpersonal reality of objects, 
events, and people. In basic cognitive terms, it may be 
that consciousness has a close relationship to what lin¬ 
guists call deixis, the realm of things that can be 'pointed 
to'. A number of linguistic expressions serve the process 
of deixis, which may be a fundamental aspect of human 
socially shared reality. Deixis includes the perceptual 
world, but also the social world, even the shared imagin¬ 
ing of a nonvisible world. Children can talk about Santa 
Claus and the tooth fairy in conventional ways. 

Philosophically, consciousness may have a role in 
objectification - creating a world of entities that have 
known or agreed-on properties, and that persist over 
time. It is significant that the dorsal stream of the vis¬ 
ual cortex does not directly lead to conscious outcomes 
even though it is essential for both spatial localization 
of the self and for actions like reaching. Therefore it is 
not the case that all visual events can be objectified, or 
that they can necessarily become objects of conscious¬ 
ness. Indeed, we can make the case that it is important 
that the dorsal and ventral visual stream must be dis¬ 
sociated from each other in order to prevent interfer¬ 
ence between temporally precise visuomotor actions. 

Objectification - turning events into stable objects 
of perception or thought - is also a crucial part of 
human thinking. Humans routinely commit the logi¬ 
cal fallacy of reification, turning an abstraction like 
'democracy' into a hypostatized entity. That mental 
process has its uses, but it can also be misleading. We 
also routinely freeze dynamic events, like aspects of 
economics, biology, or physics, which might be dealt 
with more accurately with mathematical tools for 
dealing with fast-changing entities. Newtonian calcu¬ 
lus was invented precisely to describe dynamic events 
that can be labeled by words in only a very impover¬ 
ished fashion. Since the invention of calculus, numer¬ 
ous mathematical advances have made it possible to 
measure and describe dynamical systems (like the 
brain) with greater accuracy. However, human lan¬ 
guage and perception have not kept pace. We still talk 
about 'the economy' or 'the brain' in essentially static 
terms. 

It is at least possible that excessive reification 
reflects our conscious style of perceiving and thinking 
about the world. We may be biologically condemned 
to 'ventral stream thinking', in spite of the fact that 
our mathematics are able to deal with a more dynamic 
world than most of us do on a daily basis. 

These questions are becoming answerable in cogni¬ 
tive neuroscience. It is important therefore to ask them 
as clearly as we can, and not to take the reportability 
of conscious events for granted. 
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6*5 Evidence for unusual states 

The three main states of the daily cycle may give us 
a better basis for thinking about unusual states - 
drug-induced, disordered, or states of mind that may 
be reached after years of training, such as meditation 
or yoga. Human history and literature contain numer¬ 
ous descriptions of extraordinary states of mind, but 
scientists set high standards for extraordinary claims. 
Some methods, like neurofeedback and hypnosis, 
are known to have robust effects, but we still need to 
know much more about their brain bases and results. 
It is possible to err on the side of too much skepticism 
as well as too much credulity; good scientists aim to be 
open-minded, but still look for very strong evidence. 

Some mystical experiences traditionally are asso¬ 
ciated with epilepsy, as in the famous case of Fyodor 
Dostoyevsky, who suffered from epilepsy and described 
powerful mystical 'auras' before seizures. Indeed the 
first book on epilepsy was attributed to the Greek 
physician Hippocrates (fourth to fifth century BC), 
titled. On The Sacred Disease. Epileptic patients com¬ 
monly describe altered states of consciousness in asso¬ 
ciation with EEG hypersynchrony, even without overt 
seizures (Figure 8.5). Most of those altered states are 
experienced as unpleasant. Since hypersychrony alters 
the workings of the thalamocortical core, there could 
well be a link between altered subjective states and 
brain rhythms. 

Psychedelic drug effects have been compared to wak¬ 
ing dreams, often showing vivid visual hallucinations, 
delusional beliefs, dream-like actions, intense emotional 
encounters, difficulty keeping track of time, and sharp 
discontinuities from 'scene' to 'scene'. Similar experi¬ 
ences are described with a variety of psychoactive drugs, 
including LSD, cannabis, and traditional plant prepara¬ 
tions like ayahuasca. There is a structural resemblance 
between the LSD molecule and the major neuromodu¬ 
lator serotonin, which is known to be involved in REM 
dreaming. However, the link between psychedelics and 
normal dreaming is still speculative. 

Delusions and hallucinations are defining symp¬ 
toms of psychosis, which can be very harmful indeed. 
Typically the first episode of schizophrenic disorder 
occurs in late adolescence or early adulthood, often a 
period of major life change and stress. It is believed 
that marijuana use may expose some vulnerable indi¬ 
viduals to a higher risk of developing schizophrenia. 
However, in spite of decades of research, schizophre¬ 
nia continues to be an unsolved puzzle. 

One major difference between psychosis and recrea¬ 
tional use of psychedelics is the degree to which highly 


unusual experiences are desired or not. 'Recreational 
drugs' by definition are taken with the goal of having 
unusual experiences. In the psychoses, delusions and 
hallucinations are often extremely upsetting, frighten¬ 
ing, and unwanted. They occur at unexpected times, 
interfere with normal life, and can develop into pain¬ 
ful persecutory voices and distressing beliefs about 
oneself. They may be chronic. The degree to which 
these experiences are unwanted is one major difference 
between psychedelic drugs and dysfunctional psy¬ 
chotic experiences. By analogy, some people enjoy par¬ 
achuting from airplanes or high suspension bridges. 
At the right time and place, they do those things for 
pleasure. If they were forced to do them against their 
will, and in a way that feels out of control, the same 
experiences might well be shocking and traumatic. 
Mental disorders are not a matter of choice, and peo¬ 
ple who suffer from them cannot stop at will. 

In contrast, certain extraordinary experiences have 
been actively pursued over thousands of years. To expe¬ 
rience altered states humans have used a huge variety 
of methods, including psychoactive plants, long fasting, 
alcohol intoxication, years of physical and mental exer¬ 
cises, visualization methods, deliberate disorientation, 
high altitudes, hypoxia, self-mutilation, intense sexual 
practices, dramatic settings and rites, lucid dreaming, 
rhythmic dancing, whirling, chanting and music, hyp¬ 
notic suggestion, dissociative traumas, sleep depriva¬ 
tion, and social isolation. Many of these practices are 
no better understood today than they were centuries 
ago. Traditions like Buddhism and Vedanta Hinduism 
believe that humans can and should change their states 
of consciousness and their sense of self. That belief can 
be found in many times and places. 

Self-reports about altered states are not persuasive to 
skeptics. Simply asking people whether some practice 
make them happier or more relaxed does not make for 
convincing evidence. Sugar pills can also lead to glow¬ 
ing testimonials. Indeed, there is hardly any practice 
that does not produce enthusiastic endorsements from 
believers. This is not to dismiss what may be truthful 
accounts by sincere and insightful practitioners. It is 
simply that unverifiable reports about private experi¬ 
ences have not led to cumulative progress in science. 

Brain evidence about unusual states may be more con¬ 
vincing to skeptics because people cannot control their 
brain states at will. We cannot alter our EEG or fMRI 
scans to prove a favorite hypothesis. Obviously some 
psychological tasks are also resistant to personal biases. 
But the vast majority of studies of meditation and related 
techniques are seriously weakened by self-report biases. 
These results are rarely believed by scientific skeptics. 
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6.5 .1 Hypnosis , conversion disorder , and brainwave 
entrainment 

About one-fourth of the normal population is highly 
hypnotizable, as assessed by a standard hypnotic 
induction procedure. Hypnotic suggestions can change 
brain events, as in reducing the event-related potential 
(ERP) in response to a flashing light simply by 'hallu¬ 
cinating' a cardboard box covering the light (Spiegel, 
2003). We do not know why hypnotic induction is 
effective in so many normal people. About 5% of the 
population are said to be 'hypnotic prodigies', who can 
hallucinate perceptual events, such as the buzzing of a 
fly; they can also enter into apparently dissociated per¬ 
sonality states (Hilgard, 1974). fMRI studies show that 
hypnotically suggested pseudo-pain activates some of 
the same brain regions as real pain, and on the posi¬ 
tive side, that hypnotic procedures can alleviate pain 
in chronic pain patients who have no other recourse 
(Spiegel, 2003). 

The brain correlates of hypnosis itself may finally 
be coming into focus. For two centuries scientific stu¬ 
dents of hypnosis have known that hypnotizable indi¬ 
viduals may experience a mild sense of loss of control 
over their voluntary movements. One common hyp¬ 
notic induction method is to tell subjects to allow their 
arms to float upward freely. Good hypnotic subjects let 
that happen without discomfort. It is possible there¬ 
fore that hypnosis may involve a functional dissocia¬ 
tion between the part of the brain that's involved in 
voluntary control (such as the dorsolateral prefrontal 
cortex, DL-PFC) and the ability to monitor cognitive 
conflict (the anterior cingulated cortex, ACC). 

Egner et al. (2005) reported in an fMRI-EEG study of 
hypnosis in the Stroop task that 'hypnosis decouples 
cognitive control from conflict monitoring processes of 
the frontal lobe'. Along similar lines, Faymonville et al. 
(2006) reported a widespread network of activations 
associated with hypnosis, but a relative deactivation 
of the precuneus, a brain region that is believed to be 
involved with the perspective of the self. That result 
might be consistent with a dissociation between self- 
control and self-monitoring during hypnosis, espe¬ 
cially in highly hypnotizable subjects. 

We are also seeing a scientific return to the topic of 
conversion disorder, called hysteria in medical history. 
In conversion disorders people experience 'psycho¬ 
genic' analgesia, hyperalgesia, imagined symptoms of 
medical disorders, paralysis of the hands or arms, and 
the like, all in the absence of identifiable nerve dam¬ 
age. Because conversion symptoms were found to be 
changeable and responsive to suggestion, nineteenth 
century physicians proposed that the problem was 


due to autosuggestion. That is still a viable hypothesis, 
although it does not follow necessarily that conversion 
symptoms can be treated by suggestion. 

Mild conversion-like symptoms are extremely com¬ 
mon. Medical students who are first being exposed 
to the physiological details of serious diseases often 
become worried when they seem to notice 'real' symp¬ 
toms in themselves. That 'medical student syndrome' 
is very common and it generally fades over time. It is 
possible that conversion disorders are a more lasting 
and disabling version of the general human tendency 
toward autosuggestion. The placebo effect is a positive 
version of the same phenomenon. 

The 'hysteria' diagnosis has been strongly criti¬ 
cized in part because it was thought to be limited to 
women. However, nineteenth century physicians like 
Jean-Paul Charcot and Pierre Janet emphasized their 
findings that men showed conversion symptoms, 
especially after severe trauma like a railroad accident. 
Nevertheless, the very concept of conversion disorder 
lost popularity for decades. In recent years carefully 
controlled brain imaging studies have revived inter¬ 
est; there really is a patient population that suffers 
from these problems, which can be very painful and 
disabling (Cojan et al., 2009). Getting a clear diagnosis 
is a step forward, to distinguish conversion impair¬ 
ment from genuine brain damage. 

6.5.2 Neurofeedback 

Neurofeedback is defined as learning to control brain 
events by giving sensory (conscious) feedback, con¬ 
tingent on some brain event. In the animal literature 
it is often described as 'operant conditioning', which 
is really an experimental procedure rather than an 
explanatory framework. Neurofeedback is one kind 
of biofeedback training, which also includes training 
of peripheral events like increasing the warmth of a 
finger. Simply by holding a thermometer showing 
small changes in finger temperature one can learn to 
increase peripheral warmth, for example, which seems 
to help people relax and lower blood pressure. 

Biofeedback studies of animals, using operant 
conditioning, have shown positive results over sev¬ 
eral decades. Animals with implanted electrodes can 
be trained to increase the activity of single neurons, 
and of larger populations of neurons. When justified 
for medical reasons similar training effects can be 
shown in humans. It is important just to stop and con¬ 
sider what a remarkable result that is. In a brain with 
billions of neurons, simple sensory feedback can allow 
training of voluntary control over otherwise entirely 
involuntary neuronal firing and other brain events. 
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However, neurofeedback training for specific brain 
oscillations, for example, may not result in enhance¬ 
ment of the targeted event. Surprisingly, even 'alpha 
neurofeeback training', for example, does not neces¬ 
sarily result in proven increments in alpha wave activ¬ 
ity This is a disconnect in the literature, which has 
mostly focused on testing practical outcomes. Oddly 
enough, the 'feedback' aspect of neurofeedback is 
sometimes lost. 

This becomes less of a paradox if we consider that 
brain rhythms are under precise homeostatic control 
(John, 2002). If neurofeedback works, it is not just flip¬ 
ping a light switch in the brain; rather, we are jumping 
up and down on a large rubber boat floating on a lake. 
Our jumping causes real changes, but the brain still 
stays afloat - which is a very good thing. The brain 
is complex, dynamic, and multistable. That is how 
living systems can survive over many kinds altered 
environments. 

We therefore have two separate questions: 

• Does brain feedback show results? After many 
years of study the answer seems to be a very clear 
'yes'. Figure 8.51 shows an example of fMRI feed¬ 
back effects in chronic pain patients. 

• Does neurofeedback have a simple, direct effect 
on the brain, or does it have both direct and indi¬ 
rect effects? The evidence seems to favor the latter, 
including important positive effects in conditions 
like ADHD, epilepsy, depression, and perhaps oth¬ 
ers (Gruzelier, 2008) (Figure 8.52). 

A large number of studies of EEG neurofeedback 
show significant results, but long-term studies are still 
often lacking. Because neurofeedback may work in 
cases where other medical treatments have failed (as 
in depression and epilepsy), long-term trials would 
seem to be vitally important. 


6.5.3 Sensorimotor rhythm (SMR ), alpha/theta 
feedback , and other trained oscillations 

Sterman and Egner (2006) describe evidence that 'neu¬ 
rofeedback treatment of epilepsy/seizure disorders 
constitutes a well-founded and viable alternative to 
anticonvulsant pharmacotherapy'. That is important 
both medically and scientifically. Drug treatments 
for epilepsy do not always work, and having another 
treatment modality available is helpful for many 
patients. Scientifically, because epilepsy (which can 
have many different causes) manifests as slow EEG 
hypersychrony, the ability to modify seizure activity 
in the brain tells us something important about the 
role of brain rhythms and their voluntary control. 

What is the SMR, the sensorimotor response? 
SMR was discovered in EEG studies of cats that were 
taught to suppress a previously learned and reinforced 
response. These animals shows a transient, alpha-like 
EEG rhythm in the sensorimotor cortex. 'This rhythm 
was characterized by a frequency of 12-20 Hz, not 
unlike EEG sleep spindles, with a spectral peak around 
12-14 Hz (Roth et ah, 1967). The investigators decided 
to study this distinct rhythm directly, attempting to 
apply the operant conditioning method to see if cats 
could be trained to voluntarily produce SMR, by mak¬ 
ing a food reward contingent on SMR production. Cats 
easily accomplished this feat of EEG self-regulation, 
and the behavior associated with SMR was one of cor¬ 
poral immobility, with SMR bursts regularly preceded 
by a drop in muscle tone. When the Sterman labora¬ 
tory was commissioned to study the effects of an epi¬ 
lepsy-producing drug, they happened to discover 
that SMR-trained cats were less susceptible to the 
chemically induced seizures. This effect generalized 
to humans. Intracranial studies in animals showed 
that the SMR comes from the ventrobasal nuclei of the 



FIGURE 8.51 Local brain activity controlled by fMRI feedback. Local brain activity increases 
after feedback training for fMRI activity in chronic pain patients. The training target was activity in 
the rostral anterior cingulate cortex, which is known to be involved in pain perception (rACC). Source : 
deCharms et al, 2005. 
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thalamus, and that it has a major role to play in inhibi¬ 
tion of unwanted responses. 

SMR feedback has now been shown to be an effec¬ 
tive treatment in cases of human epilepsy, ADHD, and 
impulse control disorder (Sterman, 2006). It is there¬ 
fore a potential treatment. Thus SMR is one of the rare 
cases (so far) where neurofeedback has resulted in a 
thorough psychobiological understanding of an impor¬ 
tant medical condition. With improved brain recording 
methods we can expect to see more of such encour¬ 
aging results. (Similar findings have been reported 
with another endogenous brain oscillation, the 'Slow 
Cortical Potential'.) 

According to Sterman (2006) the American Academy 
of Child and Adolescent Psychiatry considers neurofee- 
back for seizure disorders to meet criteria for 'Clinical 
Guidelines' of evidence- based treatments '...these 
results have been achieved in an extremely difficult sub¬ 
group of epilepsy patients, those with poorly controlled 
seizures who had proven unresponsive to pharmaco¬ 
logical treatment. On average 80% of patients trained at 
enhancing SMR amplitudes were shown to display sig¬ 
nificant clinical improvements'. (Kaplan, 1975; Seifert & 
Lubar, 1975; Sterman, MacDonald, & Stone, 1974) 

It is worth pointing out that neurofeedback is inher¬ 
ently still mysterious in that we do not understand 
the role of conscious feedback in enabling voluntary 
control of thalamcortical waveforms. That reflects 
our current limitations. SMR represents an outstand¬ 
ing example of solid scientific foundations enabling 
important advances in treating serious disorders. It 
would seem to be especially important to study SMR 
for attentional deficits and impulse-control disorders, 
which are often difficult to treat. 

6.5.4 Rhythmic entrainment of brain waves 

If brain rhythms constitute one basic code for neural 
signaling, it would be interesting to know if we could 
drive them externally. Brain wave entrainment has been 
described with TMS (transcranial magnetic stimulation) 
and tACS (transcranial alternating current). In neuro¬ 
surgery direct electrical stimulation of the cortex has 
been used for half a century. (See Chapter 1, and earlier 
in this chapter.) It might also be possible to use audi¬ 
tory, visual, or motor rhythms. However, although there 
are many popular claims about the effects of auditory 
entrainment, they have not been demonstrated to work 
by appropriate medical and scientific standards so far. 

Ideally, an external sensory driver for brain waves 
would have to be aimed at the right part of the brain, at 
the right rhythm, and to be started and stopped at the 


right time. There are still many unanswered questions 
about endogenous brain rhythms and how they interact 
with external inputs. As we have mentioned, visual 
stimulus detection is better at the peak of the alpha wave 
than at the trough (Vanrullen & Koch, 2003). That may be 
only the first example of how the ongoing activity of the 
brain shapes input processing. The brain is always active, 
and sensory input modulates only ongoing activities. 

Neurofeedback and techniques like acupuncture 
also have been studied in animals, thereby ruling out 
placebo effects. But so far, what seem to be promising 
methods have not been widely applied. We now under¬ 
stand more about brain oscillations, for example, which 
gives us a better rationale for neurofeedback training. 
But we are just beginning to understand the details. 
It is worth noting that neurofeedback training always 
involves a conscious feedback signal (Baars, 2002). 

Hypofrontality - decreased metabolism in the fron¬ 
tal lobes may be a common theme for altered states. 
Dietrich (2002) argues that lowered frontal metabo¬ 
lism is a shared feature of hypnotizability, 'runner's 
high', 'flow', and other unusual states of mind. Low 
frontal metabolism also occurs in simple drowsiness 
and other semiconscious states, as in emerging from 
general anesthesia. 'Grogginess' is a commonsense 
term for the state in which we are awake but not very 
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FIGURE 8.52 Alpha/theta training improves high-level musical 
and dance performance. Gruzelier (2008) has shown that 20 neurofeed¬ 
back training sessions designed to increase alpha and theta activity 
had marked effects on performance in conservatoire musicians and 
dancers. Evaluation was by independent judges who were blind to the 
treatment conditions. Neurofeedback training (NFT) showed improve¬ 
ments equivalent to two conservatoire grades, while other treatments 
showed no difference. Technical skill levels did not improve, but musi- 
cality, conviction, stage presence, and relaxation were improved. 
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alert, and executive functions may seem impaired. 
Obviously not all these states are the same. Low fron¬ 
tal metabolism may be one overlapping feature. 

6.5.5 Meditation and similar practices 

Meditation is often described as a state of relaxed 
alertness. The term 'meditation' covers a host of men¬ 
tal techniques with possibly different effects, but one 
common method is silent repetition of a word called a 
mantra. There is unresolved debate whether the man¬ 
tra can be any arbitrary sound or not. Mantra medita¬ 
tion is described as a method for changing mind states, 
as taught in Asian traditions like Vedanta, Buddhism, 
and traditional Chinese medicine. It has also been 
widely practiced in European and Middle Eastern tra¬ 
ditions. Because it is so simple (no equipment required) 
it is almost sure to have been tried in many times and 
places, not excluding hunter-gatherers, who often have 
elaborate shamanistic initiations and practices. 

A different procedure is called 'mindfulness medi¬ 
tation', described as '... allowing any thoughts, feel¬ 
ings, or sensations to arise while maintaining a specific 
attentional stance: awareness of the phenomenal field 
as an attentive and nonattached observer without 
judgment or analysis' (Cahn & Polich, 2006, 2009). 
These two techniques are the best-studied ones so far, 
with hundreds of articles in peer-reviewed journals. 
They may differ in the degree to which they encourage 
free-floating attention versus focused concentration on 


a single mental content. Over several decades, medi¬ 
tation methods often have been reported to increase 
brain activity and coherence (phase-locking), espe¬ 
cially in the theta, alpha, and the beta-to-lower-gamma 
band of the EEG. Frontal-lobe interhemispheric coher¬ 
ence is also reported, as well as improved attentional 
functioning (Lazar et ah, 2000; Lutz et ah, 2004; Tang 
et ah, 2007). 

One of the surprises with simple mantra repetition 
is a significant drop in metabolic activity, even in new 
practitioners, as reflected in 'breath suspensions' - 
spontaneous stopping of the breath, but without com¬ 
pensatory breathing afterward (Figure 8.53). This 
is different from holding our breath voluntarily. In 
swimming, for example, we may take a deep breath 
before diving and then feel the need to take some 
extra breaths after coming up for air. Normal breath¬ 
ing is under tight homeostatic control, and voluntary 
breath-holding requires compensatory breathing, to 
make up for decreased oxygen. Therefore the finding 
of spontaneous, uninstructed breath suspension with¬ 
out compensatory breathing is a genuine surprise. It 
does not occur in normal resting controls. The lack of 
compensatory breathing suggests that energy demand 
has indeed dropped more than an ordinary resting 
state, as has been demonstrated by measuring 0 2 / C0 2 
exchange (Benson et ah, 1975; Hoffman et ah, 1982). 

Herbert Benson and colleagues have proposed 
that these results represent a 'relaxation response', 
on the analogy with other physiological reflexes. The 
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FIGURE 8.53 Spontaneous breath suspension in mantra repetition. Repeated, spontaneous 
breath suspension in silent mantra repetition coincides with subjects' reports of 'pure conscious' expe¬ 
riences. Breath suspensions occur without compensatory overbreathing, different from voluntary 
breath-holding, which tends to be followed by compensatory breathing. Breath suspensions are said 
to signal moments of 'pure consciousness', defined as clear conscious moments without content. These 
conscious moments are the purpose of the technique. This study was performed on practitioners of 
Transcendental Meditation, a silent mantra technique from the Vedanta tradition in India. However, 
similar phenomena may occur with the use of arbitrary mantras. Figure adapted from Benson et ah, 
1975, and Hoffman et al., 1982. 
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parasympathetic nervous system is involved with 
relaxation (in competition with the sympathetic nerv¬ 
ous system) and it is plausible that there could be a 
way of evoking a parasympathetic 'switch', for restor¬ 
ing physiological and mental resources. However, 
many other variables coincide with the effects of man¬ 
tra meditation, and it is puzzling why mere mental 
word repetition would evoke such a pervasive physi¬ 
ological and brain response. What is perhaps most 
important about the rapid physiological effects of 
mantra meditation is that is tells us there is something 
surprising and different about such a simple practice. 
The apparent metabolic drop also casts a different 
light on the brain wave results that are reported to go 
along with it. 

Spontaneous breath suspensions are also reported to 
be associated with reports of periods of 'pure conscious¬ 
ness', defined as relaxed alertness without any specific 
mental contents. The aim of mantra meditation is to pro¬ 
duce 'pure consciousness' experiences until they become 
habitual and are integrated with daily life. These effects 
have been described with novices and advanced practi¬ 
tioners, and also with the use of arbitrary mantras as in 
the Relaxation Response. Converging evidence has come 
from measures of metabolism, sympathetic nervous sys¬ 
tem tone, and recently, very large-scale changes in gene 
expression. Because the exact functions of 'relaxation- 
related' genes are not yet clear, this promising direction 
requires additional studies (Jacobs et al., 1996; Dusek 
et al., 2006). 

What brain changes take place after many years of 
pursuing these practices? EEG studies of advanced 
meditators have shown high levels of gamma, alpha, 
and theta synchrony (e.g. Lutz et al., 2005). These stud¬ 
ies commonly focus on highly trained individuals. 
For example. Lutz and colleagues studied four 
advanced meditators with 15,000 to 45,000 hours of 
training. Advanced practitioners must be different in 
many ways. They may resemble top athletes, dancers, 
or musicians, who may begin early in life with excep¬ 
tional talents and predilections. The result it essen¬ 
tially like asking why Tiger Woods is an outstanding 
athlete. That question is inherently difficult, because 
we would have to start from childhood with control 
subjects, and do longitudinal studies over years, only 
to end up with a single case history. 

Nevertheless, it is still interesting when measurable 
brain changes can be observed over half an hour of 
meditation, even among exceptional individuals. Broca 
and Wernicke used very small numbers of rare subjects 
to discover the language areas of the cortex in the nine¬ 
teenth century (Chapter 1). Intensive study of a few 
patients also led to major progress in understanding 


the hippocampus in memory (Chapter 1). There are 
good precedents for focusing on small numbers of 
exceptional individuals, as long as the evidence cannot 
be influenced by biases and expectations. 

However, it is almost impossible to find control 
groups that are comparable to advanced practition¬ 
ers. As a result, it is difficult to know how well these 
results generalize to a wider population. The answer 
to that question historically is that over time, the gen¬ 
erality of the findings can be tested in other ways. 
Broca's area is really involved in speech production, as 
we can now tell in dozens of different ways, and with 
tens of thousands of human subjects. But it all started 
with careful studies of a single patient. The case of HM 
demonstrates even better methodology, because this 
surgically lesioned man was studied over decades by 
many different laboratories. That ensured that differ¬ 
ent researchers were all dealing with the same human 
being. However, that kind of coordination has not yet 
been achieved in studies of altered states. As a result, 
we have numerous articles on what may be vastly dif¬ 
ferent advanced practitioners. 

High interregional coupling in the theta-alpha 
range is suggested as an explanation for meditation 
states (Hebert, 2005). Advanced practitioners of medi¬ 
tation and similar training techniques may describe 
states of high tranquility, consciousness without spe¬ 
cific contents, expansion of subjective space, con¬ 
sciousness during sleep, and the like. Researchers are 
only beginning to seriously address the subjective 
experiences of advanced practitioners, which is after 
all what motivates meditators to do what they do. 

As a result, even after four decades of studies, we 
do not know nearly as much about meditation effects 
as we would like. By comparison the neurofeedback 
literature is in somewhat better shape, although it, 
too, still has numerous gaps. None of this research 
literature is entirely satisfactory; none has been 
studied quite as much as we might like. One hope¬ 
ful sign is a better understanding of EEG rhythms 
in specific brain locations, identified with fMRI. 
Brain rhythms historically have been merely under¬ 
stood as correlates of something interesting, with no 
understanding of what exact role they might play. 
High alpha coherence is a little like knowing that 
people have heartbeats, but without understanding 
the role of blood circulation and the heart's pump¬ 
ing action. Fortunately the lonesome tree of unin¬ 
terpreted brain rhythms is now being enriched by a 
young forest of related facts and plausible functions. 
Improved understanding allows us to make better 
sense of the brain correlates of meditation and similar 
practices. 
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6.5.6 Out of body experiences can be induced by 
brain stimulation 

Some of the most reliable results come from studies of 
'out of body' experiences (OBEs). Seeming to look at 
one's body from the outside certainly counts as extraor¬ 
dinary. On this topic there has been genuine progress. 
We now know, for example, that direct stimulation of the 
right posterior parietal cortex (PPC) can evoke dramatic 
changes in experienced body space (Figure 8.54). Some 
epileptics experience major alterations in body space, 
perhaps because of hypersynchronous activity affecting 
this region. Damage to the right PPC is also known to 
produce alterations in body space. Parietal neglect is a 
well-described disorder that leads to radical alterations 
in perceived space. 



FIGURE 8.54 The right parietal cortex and out-of-body expe¬ 
riences. Tong (2003) after Blanke (2002). 


Indeed, the parietal cortex has emerged as a player 
in a number of unusual experiences. It seems that we 
can begin to see a growing body of evidence that bears 
on at least this one kind of extraordinary experience. 


7.0 SUMMARY 


Much territory has been covered in this chapter. The 
good news is that we are getting a better understanding 
of consciousness and attention. There is even a major 
simplification of the evidence, as the pieces of the puz¬ 
zle start to come together. 

Attention and consciousness can be considered as 
complementary processes (see Figure 8.55). Conscious 
contents often are thought to involve the widespread 
distribution of focal contents, like the sight of a rab¬ 
bit. As soon as we see a rabbit (consciously), we can 
also judge whether it's real, if it's somebody's pet, we 
can laugh or scream for fear of being bitten, and try 
to remember if rabbits carry rabies. The point is that a 
great variety of brain events can be evoked by a con¬ 
scious object, including a great variety of unconscious 
processes. Conversely, we can focus cognitive resources 
on any topic to which we decide to devote attention, 
such as the contents of this book. In order to learn, we 
direct attention to a topic over and over again, with the 
result that we have many different moments of con¬ 
sciousness, each one able to evoke additional cogni¬ 
tive processes, as needed. Learning a difficult subject 
involves allocating attentional resources over time. 

Consciousness typically involves a small amount of 
content at any given moment, like the sight of a rose 
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FIGURE 8.55 Attention and consciousness. Attention often is thought of by analogy to a spotlight, moving 
to 'point at' and select some part of the world for conscious processing. Controlling the 'spotlight of attention' 
requires frontal and parietal cortex as well as subcortical regions. Recent evidence suggests that perceptual con¬ 
sciousness may activate cortical regions in the opposite direction, beginning from primary sensory cortex (pos¬ 
terior) and activating more frontal regions. Thus we can think of two streams of interacting information flowing 
back and forth in the cortex, one selective (selective attention), and the other integrative and distributive in its effects 
(perceptual consciousness). Source : Baars & Fu, 2007. 
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bush, which may recruit and mobilize many uncon¬ 
scious brain functions. In attention, major functions 
like motivation, planning, emotional needs, and the 
like, become directed to a specific object in the world. 
Voluntary attention particularly requires the recruit¬ 
ment of frontal lobe resources toward acquiring new 
and better information on behalf of some explicit goal 
or task. Frontoparietal regions of cortex may then 
enhance processing in visual regions (for example) in 
order to sharpen, clarify, and make conscious some 
part of the visual world. Episodic and declarative 
learning seem to occur automatically as a function of 
conscious contents, which in turn may be guided by 
voluntary attention. The three major thalamocortical 


brain states of the circadian cycle show different oper¬ 
ating styles of the attention-consciousness dialogue. A 
major function of slow-wave sleep and REM dreams is 
to consolidate the memory of events that were initially 
perceived and learned in the waking state. 

Understanding the brain basis of consciousness and 
attention is still a formidable challenge. Some hints come 
from Table 8.4, describing what the brain is doing in the 
three basic states. Notice the role of neuromodulation, 
chemicals spread by small numbers of neurons located in 
basal nuclei, which project their axons to higher parts of 
the brain. The thalamus also plays an essential role in the 
three major states. However, the contents of conscious¬ 
ness are believed to be largely controlled by the cortex. 


TABLE 8.4 Brain activity in waking, dreaming, and sleep 


Thalamocortical 
operating mode 

Raw EEG (Surface 
electromagnetic field) 

Neurons and their 
interactions 

Other features 

Waking consciousness 

People report normal 

conscious experiences, can 
carry out standard cognitive 
tasks, and exercise voluntary 
(executive) control over 
their skeletal muscles. 

Fast, low-amplitude, irregular 
waveforms, which appear 
to contain many different 
frequency bands. Some highly 
synchronized activities may 
be happening even while the 
surface EEG looks mixed. 

Highly active and interactive 
with other thalamic and 
cortical neurons. 

Free flowing sensory input and motor 
output, with limited-capacity 
competition for conscious and 
executive events. Active problem¬ 
solving and learning. Conscious 
episodes are automatically encoded in 
memory by the hippocampal complex 
and neocortex. Some episodic 
memory consolidation may occur 
during waking, but mostly in sleep. 

REM dreaming 

When questioned, reported as 
conscious. 

Fast, low-amplitude, irregular 
waveforms, which appear 
to contain many different 
frequency bands. Some highly 
synchronized activities may 
be happening even while the 
surface EEG looks mixed. 

Active, highly interactive with 
other thalamic and cortical 

neurons. 

Blocked sensory input and motor 
output. 

Conscious contents are internally 
generated. Tacit problem-solving 
may occur. 

It is believed that implicit and 
procedural memories may be 
consolidation in the REM state. 

Sleep stages 1-4 

Arousability goes down from 
stage 1-4, and less mentation 
is reported. However, some 
mental activity is reported 
even from deep sleep. 

As arousability goes down, 
there is an increasing 
number of slow, high- 
amplitude, regular waves. 

Massive 'buzz-pause'activity 
in near unison in the 
thalamocortical system. The 
simultaneous 'buzz' activity 
(the Up State) allows for 
waking-like interactivity. The 
'pause' or Down State blocks 
signaling between cells. 

Blocked sensory input and motor 
output. 

Recent episodic memories are replayed 
in the hippocampal-neocortical 
system during the Up States of the 
slow-wave rhythm, and explicit 
memory consolidation occurs. 

Epileptic loss of consciousnes 


Slow, high-amplitude, spike¬ 
shaped, regular. May originate 
in the hippocampus and 
temporal lobe, though other 
sources have been reported. 

Massive 'buzz-pause'activity 
in near unison, thalamic and 
cortical. Hypersynchronous 
activity is thought to recruit 
expanding areas of cortex. 

Lost functions, including 

consciousness, may be partial or 
complete. Automatic actions may 
occur, resembling sleepwalking. 
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Table 8.4 summarizes the major features of the three 
daily states of consciousness. Notice that the brain 
activity (although most important) is not the only 
aspect of sleep and waking states. Prominently the sen¬ 
sory input and motor output is blocked during REM 
dreaming (to prevent us from acting out our dreams) 
and during slow-wave sleep. In some REM sleep dis¬ 
orders, motor inhibition at the spinal cord is released, 
with the result that people may act out hallucinatory 
or delusional behaviors in a semiconscious way. Some 
sources suggest that thalamocortical disruption of the 
normal rhythmic cycle may be involved in serious 
brain disorders like psychosis. 

The slow wave of sleep can be decomposed into an 
'Up State' (the peak of the slow wave) and a 'Down 
State' (the trough). The Up State allows neurons to 
fire at their normal waking rates, and to interact with 
each other for a second or less. The Down State blocks 
these activities. Thus, some scientists have suggested 
that we may actually be awake about one second at a 
time during SWS (slow-wave sleep). When we look at 
the arousal threshold on this second-to-second scale, 
therefore, the arousal threshold may in fact oscillate 
quickly back and forth. 


Like the waves in the ocean, signal propagation in the 
brain is a mix of many different waves. Alpha waves were 
the first to be observed, because they are so easily spot¬ 
ted: They appear in the raw EEG above the occipital cor¬ 
tex when the eyes are closed in a relaxed state. When the 
eyes are opened, alpha typically is interrupted by faster, 
irregular, low-amplitude EEG, labeled beta and gamma. 
However, complex waveforms can be decomposed into 
separate wave bands. These have now extended into well 
up to 200Hz, sometimes lOOOHz.The conventional range 
for gamma is still taken to be 30 to 120 Hz, but it is safer 
just to use numbers. Our verbal labels for brain rhythms 
are arbitrary and likely to change. 

Waveforms in the brain interact. Waking rhythms 
may add, subtract, synchronize, desynchronize, phase 
lock, and multiplex. Slower waves may carry (multi¬ 
plex) faster waves. It has been suggested that the very 
slow oscillations (0.1-0.5Hz), delta (0.5-3 Hz), theta 
(4-7 Hz), alpha (8-12), beta (13-29), and gamma (30- 
120) all may be multiplexed under some conditions. 
Certainly theta commonly carries gamma activity, 
and the Up State of the delta wave during slow-wave 
sleep involves waking-like complex, interactive, high- 
frequency activity. 


8*0 STUDY QUESTIONS 

1. What major brain structures are involved in 
maintaining conscious waking, deep sleep 
(slow-wave sleep), and REM dreams? How do 
we switch between states? 

2 What is the best-established function of slow- 
wave sleep? Of waking? Of REM dreaming? (If 
no function is established, just say so.) 

3 When you learn to ride a bicycle really well, 
what happens to your cortical activity for sen¬ 
sorimotor control? 

4 How can two brain areas that represent the vis¬ 
ual field be coordinated? 

5 What is meant by multiplexing of brain waves? 
What might be an example? 

6 What are the main effects of selective attention? 
What part of the brain appears to control pur¬ 
poseful attention? 

7 What's meant by 'the architecture of sleep'? 
Describe some of its features and timing. 


8 What are some hypotheses about REM dreams? 
What kind of function might it have? How 
widely is REM dreaming distributed in the ani¬ 
mal kingdom? 

9 What are the basic functional and electrophysi- 
ological features of the conscious waking state? 
What kind of cortical waveforms seem to occur 
during conscious waking? 

10 Describe the Up and Down States of slow 
oscillations. 

11 Describe neurofeedback training. What is it use¬ 
ful for? 

12 Is there convincing evidence that hypnosis reflects 
a genuine brain state? 

13 What are the pros and cons of surface (scalp) 
EEG compared to intracranial EEG (iEEG)? 

14 Meditation methods have been described for thou¬ 
sands of years. What are some commonly cited 
features? What evidence do we have pro or con? 



What sortoflife (if any), what sort of world, what sortofself, can be preserved in a man who 
has lost the greater part of his memory and, with this, his past, and his moorings in time ? 


Oliver Sacks (1995) 


Exoculrvo memory Perceptual memory 



(b) 




How memory might be encoded in many parts of cortex. The neuroscientist Joaquin Fuster's concep¬ 
tion of long-term memory traces encoded in cortex. Posterior cortex has more perceptual varieties of 
stored traces, while the frontal lobe has executive and action-related traces. The principle is believed to 
be that neocortex (the visible large cortex) encodes memory by strengthening synaptic connections 
among vast arrays of neurons that perform higher cognitive functions. Source: Fuster, 2004. 
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1.0 INTRODUCTION 

Memory can be defined as a lasting representation that 
is reflected in thought, experience, or behavior. Learning 


is the acquisition of such representations, which involve 
a wide range of brain areas and activities. Human mem¬ 
ory has some surprising limits and some very impres¬ 
sive capacities. For example, most students would like 
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to have a 'better memory' - by which we mean a capac¬ 
ity deliberately to store and recall information, at will, 
easily and accurately However, the human brain has 
not evolved for academic study and testing, which is 
a very recent cultural invention. Rather, it has evolved 
for the tasks of survival. Our best memory performance 
is not for the exact symbol sequences that conventional 
computers handle so well. Rather, our brains are excep¬ 
tionally good at dealing with complex, ill-defined, and 
novel challenges, the kinds that people have to deal 
with in the real world. 

Humans are exceptionally flexible in adapting to 
new conditions. Learning and memory provide the 
royal road to that flexibility. Our great capacity for 
learning allows us to use a brain adapted to a neo¬ 
lithic environment and apply it with great success in 
a world of computers, brain science, and academic 
learning. There must be something special about our 
great ability to learn. 

Memory storage is believed to involve very wide¬ 
spread synaptic alterations in many parts of cor¬ 
tex. This process is often taken to involve large-scale 
Hebbian learning, following the rule that 'neurons 
that fire together, wire together' (see Chapter 3 and 
the Appendix). Thus, correlated activity between 
neurons leads to strengthened connections between 
them, both excitatory and inhibitory. Temporary cell 



assemblies are thought to maintain immediate memo¬ 
ries, while long-term memories require more lasting 
strengthening of synaptic connections, both excitatory 
and inhibitory ones. While evidence has been found 
for these phenomena, we cannot yet observe changed 
long-term connectivity directly throughout the brain 
regions where it is thought to occur. Thus, the Hebbian 
memory trace itself remains an inferential concept. 

In some views, all of the cortex may be able to learn 
by changing synaptic connectivities, from the poste¬ 
rior perceptual regions to the frontal executive and 
motor cortex (Fuster, 2004). Others focus more on the 
temporal lobe, which traditional neurology associates 
with memory functions. The most important brain 
structures we look at in this chapter are the neocortex - 
the visible outer brain - and the medial temporal lobes 
(MTL), which contain the two hippocampi and their 
surrounding tissues. Figures 9.1 and 9.2). The MTL is 
spatially complex, and you should devote some time to 
the drawing exercises at the end of this chapter to get 
an overall sense of its shape and location. 

Until recently, the hippocampus was believed to be 
mainly responsible for transferring experiences into 
memory, but better methods now implicate the entire 
hippocampal neighborhood, the medial temporal 
lobes. The MTL overlap with the ancient mammalian 
smell cortex, which has fewer than the six layers. The 



FIGURE 9.1 The medial temporal lobes and hippocampi. The memory regions are spatially complex and dif¬ 
ficult to visualize. This collage shows two perspectives. The left panel shows a coronal cross-section of the brain, with 
the medial temporal lobes (MTL) circled in red. The right panel shows the hippocampi in both hemispheres, tipped by 
red structures, the amygdalae. Notice that the hippocampi are looped structures, nestled inside of each temporal lobe. 
The MTL includes the hippocampi and neighboring 'rhinal 7 structures (see Chapter 5). Source : Drawn by Shawn Fu. 
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neocortex, which is the 'new' cortex of modern mam¬ 
mals, consistently has six layers, one of its most dis¬ 
tinctive features. The giant neocortex and the MTL are 
in constant dialogue with each other, as we store and 
retrieve the flow of our daily experiences. 

Because of its ancient evolutionary lineage, the 
MTL has multiple functions. The hippocampus was 
first studied as a map for spatial localization in rats 
and other mammals. However, it also encodes olfac¬ 
tion, which is why parts of the MTL are called the 
'rhinal', 'entorhinal', and 'perirhinal' cortex. ('Rhinos' 
means 'nose' in Greek, as in the word rhinoceros, 
'nose horn'). 

MTL also interacts with visual area IT, the inferior 
temporal lobe (see Figure 9.2). You may remember that 
area IT has been shown to integrate high-level visual 
object information (see Chapter 6). Neurons firing in 
this region correlate with conscious visual perception 
(Sheinberg and Logothetis, 1997). Thus, the MTL is 
strategically located to take in high-level, presumably 
conscious visual information. 

Just around the corner from the MTL is the audi¬ 
tory cortex, suggesting that auditory information can 
be fed to MTL as well (see Chapter 7), and the amyg¬ 
dalae reside near the tip of the hippocampi, a major 
hub of emotional information. Thus, MTL is a highly 
interactive crossroads, well-placed for integrating 
multiple brain inputs, and for coordinating learning 
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and retrieval in many parts of the neocortex. It is a 
'hub of hubs' (Figure 9.3). 

Most of human cortex is neocortex, which bal¬ 
looned out from earlier regions over more than a 
hundred million years of mammalian evolution. As 
mentioned above, the neocortex is believed to encode 
long-term memories by altering synaptic connections 
between billions of neurons. There are literally trillions 
of such synapses in the cortex and its satellite organs, 
especially the thalamus (see Chapter 3). The hippo¬ 
campus is ideally situated to combine information 
about the cognitive (neocortex) and emotional (limbic) 
areas, and bind that information into a memory trace 
that codes for all aspects of a consciously experienced 
event (Moscovitch, 1995). 
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FIGURE 9.2 Memory areas receive visual object information. 
This midline view shows that the medial temporal lobe (MTL) is 
closely connected to area IT, the inferotemporal cortex. Area IT 
seems to support conscious visual object perception (see Chapter 
8). MTL also includes the amygdala. Auditory cortex is located 
just around the corner, on the outside of the temporal lobe. Source : 
Vuilleumier, 2005. 


FIGURE 9.3 The medial temporal lobe is a hub with wide¬ 
spread connections. The MTL, including the perirhinal cortex, has 
very wide connectivity to visual, auditory, somatosensory, emo¬ 
tional, motor, memory, and executive regions. This makes the MTL 
an ideal place to receive, bind, and distribute information for long¬ 
term memory. Notice that almost all connections run in both direc¬ 
tions (double arrows). Source : Murray and Richmond, 2001. 
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1.1 A functional overview 

Figure 9.4 suggests that the visual cortex (for 
example) first takes in the sight of a coffee cup on a 
table. Cortical activity corresponding to the perceived 
coffee cup then spreads to the MTL, which acti¬ 
vates and 'binds' widespread memory traces in both 
the visual cortex and other regions. Some scientists 
believe that, for a brief period of time, a few tenths 
of a second, the visual input regions and other parts 
of the 'bound' neocortex resonate with each other, 
as shown in coordinated gamma electrical activity 
(often called the '40 Hertz hypothesis') (Llinas and 
Pare, 1991; Engel and Singer, 2001; Freeman et al., 
2003). Comprehending a visual stimulus like a coffee 
cup probably requires several hundred milliseconds. 
Thus, in less than a second, visual cortex has identi¬ 
fied an object in front of our eyes, and triggered MTL 
to bind many regions of the neocortex to start making 
memory traces. However, as we will see, a permanent 
memory takes more time to consolidate. 

Memory is for use in the real world, and retrieval is 
therefore as important as learning. When we encounter 


a reminder of a specific past experience of the coffee 
cup, the bound memory traces Tight up' the corre¬ 
sponding regions of cortex again. We thereby recon¬ 
struct some part of the original memory, again using 
the MTL to integrate memory traces into a coherent 
conscious experience. That experience - of imagining 
yesterday's coffee cup - makes use of visual cortex 
again. Because this is the central theme of this chapter, 
we begin with a cartoon version in Figure 9.4. 

Learning is not limited to the neocortex. Other 
types of learning use other parts of the brain (see 
Section 8.0). However, in this chapter we will focus on 
learning and retrieving everyday memories. 

1.2 Learning and memory in the functional 
framework 

According to our functional framework, sensory input 
goes to 'working storage', which is part of working mem¬ 
ory which, in turn, allows information to be actively 
maintained and manipulated (Figure 9.5; see Chapter 2). 
Working memory allows us temporarily to retain small 
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FIGURE 9.4 How MTL is believed to help store and retrieve episodic memo¬ 
ries. In the left panel, the sight of a coffee cup standing on a table activates visual cor¬ 
tex up to the level of object perception (see Chapter 6). In the middle panel, storage 
is achieved when MTL coordinates widespread memory traces (involving synaptic 
modification) throughout many parts of cortex. These include semantic associates of 
the coffee cup, such as the coffee beans in the picture below. Visual features of the cup, 
like the handle, are also part of the associative complex that becomes activated. When 
the episodic memory - the sight of the coffee cup - is cued the following day, maybe 
by someone asking, "Did you like the way I made the coffee yesterday? 7 , MTL is once 
again involved in retrieving and organizing widespread cortical memory traces. Visual 
cortex is therefore needed to reconstruct the sight of the coffee cup, which is never 
identical to the original cup, but rather a plausible recreation of a pattern of visual acti¬ 
vation that overlaps with the first one. Notice that visual cortex is involved in percep¬ 
tion, learning, and episodic recall. 
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amounts of information in an accessible form. Many 
everyday tasks call upon this working memory capac¬ 
ity, such as keeping a phone number in mind for the 
several seconds it takes to dial the number. Working 
memory also gives us a sense of continuity over time, 
by embedding our immediate conscious experiences 
into a longer psychological present. There is debate 
about the exact relationships between conscious events, 
working memory, and selective attention. Some scien¬ 
tists believe that working memory (WM) gives rise to 
our conscious experiences; others suggest that working 
memory is itself coordinated by conscious cognition 
(Baars and Franklin, 2003). But there is good agreement 
that all three interact very closely. That is the most 
important point. 

Two key properties of WM are its relatively small 
capacity and its limited duration. In his famous paper 
on the span of immediate memory, George Miller 
(1956) concluded that only seven (plus or minus two) 
items can be held in immediate memory at any one 
time. More recent work suggests that the true WM 


span may be even smaller, closer to four items when 
rehearsal is prevented (Cowan, 2001; see Chapter 2). 
Likewise, the time an item is available is quite short, 
on the order of seconds. 

Why have we evolved a memory system with these 
limits? The answer is not known. One possibility 
is that WM limits give special status to only the few 
pieces of information that are most relevant to our 
present goals, thus protecting them from interference 
from irrelevant information. It might also be adaptive 
for items in working memory to fade quickly. If mate¬ 
rial lingered beyond its period of relevance it might 
block out new information. 

How we interpret and deal with material in working 
memory is determined by our current goals and con¬ 
cerns, as well as by our existing knowledge. Sensory 
and internal information may be brought to conscious¬ 
ness using attention. Once it becomes conscious, a 
number of theorists maintain that information is rap¬ 
idly encoded into long-term memory (e.g. Moscovitch, 
1990). There is also evidence for some unconscious 
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FIGURE 9.5 Explicit and implicit aspects of memory. A functional diagram for learning and memory. 
Working memory (on top) can now be viewed as input to different types of long-term memory, divided into 
explicit and implicit ones. Explicit learning and retrieval involve conscious knowledge, both for facts and autobio¬ 
graphical experiences. Memory for facts is called semantic memory, while autobiographical memory is also called 
episodic because it reflects life episodes. Working memory can manipulate explicit memories, like words, numbers, 
semantic facts, and autobiographical episodes. Implicit learning and retrieval involve primed tasks, highly prac¬ 
ticed habits, and motor skills. 
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learning, but so far only unconscious fear condition¬ 
ing has been shown to result in long-term memory 
(LeDoux, 1996). In general, conscious exposure cor¬ 
relates well with learning. While details of this broad 
picture continue to be debated, it is a useful outline to 
start with. 

Figure 9.5 brings out several features of learning 
and memory. Notice that conscious cognition leads to 
explicit learning and memory retrieval in this figure. 
An obvious example is deliberately trying to memo¬ 
rize a technical term in cognitive neuroscience. What 
may not be so obvious, however, is that implicit learn¬ 
ing also happens along with learning of conscious or 
explicit stimuli (Section 2.4). 

Thus, Figure 9.5 shows both explicit or conscious 
and implicit or unconscious learning. Episodic mem¬ 
ory is the storage of conscious episodes (also called 
autobiographical memory). Semantic memory, usually 
viewed as memory for facts, is also conscious, in the 
strict sense that people can accurately report the facts 
they believe. This is the standard operational defini¬ 
tion of conscious brain events (see Chapter 8). Finally, 
perceptual memory capacities, such as our ability to 
Team to hear' music and art, also involve conscious, 
explicit kinds of memories. 

On the right-hand side of Figure 9.5 we also see 
the learning of implicit memories. Infants may hear 
sequences of speech sounds, but they are not explicitly 
learning the rules and regularities of grammar. Those 
are apparently learned unconsciously, as we will see 
later. In general, implicit learning is often evoked by 
explicit, conscious events, but it often goes far beyond 
the events given in conscious experience (Banaji and 
Greenwald, 1995). Over-practiced habits and motor 
skills are also largely implicit. As we will see, prim¬ 
ing effects are often implicit. Contextual phenomena 
are often implicit, such as the assumptions we make 
about visual space, the direction of the incoming light 
in a visual scene, the conceptual assumptions of a con¬ 
versation, and so on. These are often hard to articulate, 
implicit, and to some degree unconscious (Baars, 1988). 

As we will see, the 'central executive' of working 
memory plays an important role in long-term learning 
and retrieval. For example, if you are trying to learn a 
word, you might deliberately rehearse it to yourself, 
using your executive capacities to control inner rehearsal 
and to shut out distractions. When you are studying for 
a test, it is a good idea to monitor your own performance 
'metacognitively' - i.e. to think about your own thinking 
process, and to see if your understanding of the material 
is good enough to pass the exam. All these are examples 
of executive processes (see Chapters 10,11, and 12). 


1.3 Implicit and explicit memory 

Implicit memory is not accompanied by conscious 
awareness that one has a memory; the memory's exis¬ 
tence is inferred only from the effects it has on behav¬ 
ior. Implicit memories may be retrieved without an 
intention to remember. Priming effects are used exten¬ 
sively to test for implicit memory. 

Priming refers to the effect of a stimulus in creating 
readiness for a similar one. For example, showing a 
picture of a face will increase the processing efficiency 
of a following face, as measured by faster reaction 
time and greater accuracy. Priming can be either per¬ 
ceptual or conceptual. 

Suppose you are asked to read a set of words and 
then your memory for them is tested a week later. We 
could test your memory directly by asking you to recall 
as many of the studied words as you can remember, or 
to recognize the words by picking them out from a list 
of old and new words. If the interval is long enough, 
you are likely to recall or recognize only a small subset 
of the items, and mistakenly classify old words that you 
studied as new ones that did not appear on the list. 

However, if your memory is tested indirectly by 
asking you to read the words as quickly as you can, 
you will be able to say old words faster than new 
words. The old words are primed. On such an indirect 
test no mention is made of memory, and the subject is 
typically not even aware that memory is being tested. 
Yet by looking at how quickly subjects read a word we 
can tell whether the previous experience left a residue 
in memory. The same result can be seen in amnesic 
patients who cannot recall studying the words at all. 

In the case of conceptual or semantic priming, 
words such as 'food' may increase the processing effi¬ 
ciency of words like 'water', even though they share 
little perceptual content. Priming can be viewed as a 
way of tapping into the general tendency of the brain 
to engage in predictive processing at every moment. 

Perceptual priming is based on alterations of per¬ 
ceptual representation in posterior neocortex associ¬ 
ated with perceptual processing. Conceptual priming 
is associated with alterations of conceptual systems in 
prefrontal cortex. 

1 . 3.1 Procedural memory 

Procedural memory refers to sensorimotor habits or auto¬ 
matic skills, which are largely unconscious. The struc¬ 
tures implicated in these habits are the basal ganglia. 

Imagine you are riding a bicycle, and you start fall¬ 
ing to the right. How would you avoid the impending 
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crash? Many cyclists say they would compensate by 
leaning towards the left, but that action would pre¬ 
cipitate the fall. When responding to the same situa¬ 
tion while actually riding a bicycle, these same cyclists 
would turn their handlebars in the direction of the 
fall. The example highlights the distinction between 
implicit and explicit knowledge. Implicit learning 
refers to the ability to learn complex information (e.g. 
skills such as bicycle riding) in the absence of explicit 
awareness. Anecdotes such as the bicycle example 
offer subjectively compelling demonstrations for 
the existence of implicit forms of knowledge that are 
distinct from (and possibly in conflict with) explicit 
knowledge .. . (Curran, 2001). 

The basal ganglia-frontal networks are the mediators 

of different classes of sensorimotor learning (Yin and 
Knowlton, 2006). 


2*0 AMNESIA 

Chapter 2 touched on the case of Clive Wearing, who 
has lived with a dense amnesia since 1985, when a 
viral infection destroyed some brain areas for memory. 
Over a few days. Wearing was transformed from a ris¬ 
ing young musician to a man for whom each waking 
moment feels like the first, with almost no recollection 
of the past, and no ability to learn for the future. 

Little has changed for Wearing since 1985. While 
he cannot recall a single specific event, some aspects 
of his memory are spared. He can carry on a normal, 
intelligent conversation. Some short-term memory is 
spared, allowing him to stay with a topic over several 
seconds. His has retained general world knowledge, 
an extensive vocabulary, and a tacit understanding 
of social conventions. Wearing also remains a skilled 
musician, able to play complex piano pieces from 
sheet music. Though he cannot remember specific 
events, he does recall a limited number of general 
facts about his life. Among the few memories that 
have survived is the identity of his wife, Deborah. He 
greets her joyfully every time she comes into his room, 
as though they have not met for years. 

However, just moments after she leaves. Wearing 
has forgotten that she was there at all. In a recent 
book, Deborah Wearing (2005) tells of coming home 
after visiting Clive at his care facility, and finding 
these messages on her answering machine: 

Hello, love, 'tis me, Clive. It's five minutes past four, 
and I don't know what's going on here. I'm awake for 
the first time and I haven't spoken to anyone. .. . 


Darling? Hello, it's me, Clive. It's a quarter past 
four and I'm awake now for the first time. It all just 
happened a minute ago, and I want to see you. .. . 

Darling? It's me, Clive, and it's 18 minutes past 
four and I'm awake. My eyes have just come on about 
a minute ago. I haven't spoken to anyone yet, I just 
want to speak to you. 

Wearing's history suggests that amnesia is selective - 
certain kinds of memory may survive while others are 
lost. Skills like the ability to speak or play the piano 
are distinct from our memories of specific events. 
Thus, memory is not unitary , but consists of different types. 
Wearing's history also hints that different parts of the 
brain may be involved in different kinds of memory. 

Figure 9.6 shows how specific Clive Wearing's 
memory loss is. Organic amnesia - the kind that 
involves damage to both MTLs - interferes with epi¬ 
sodic learning and recall. In addition, semantic learn¬ 
ing is impaired, but not semantic retrieval. Clive 
Wearing still understands the world in much the way 
he understood it in 1985; his previous semantic knowl¬ 
edge is remarkably intact. But he cannot learn new 
ideas. And he can neither learn nor remember specific 
events. We will see later why this pattern of impair¬ 
ment makes sense. 

Although all amnesic patients have memory loss, it 
varies in degree. Clive Wearing's amnesia resembles 
that of other patients, but he is unusual in his recur¬ 
rent sense that he has just awoken from some uncon¬ 
scious state. He also 'perseverates' more than most 
amnesics, repeating the same thoughts and actions 
over and over again, as in the repetitive telephone 
messages he leaves for his wife. These symptoms may 
result from additional damage to prefrontal structures 
that allow us to monitor our own actions. 


2*1 HM: the best-studied amnesia patient 

We know a great deal about Clive Wearing's life from 
the extensive publicity about his case. However, 
another patient, known only as HM, is by far the best- 
studied victim of amnesia. In the case of HM we know 
exactly where the lesion occurred (Figures 9.7 and 9.8). 
This makes him very rare. Most brain injuries are very 
'messy', can spread wider than the visible lesions, and 
may change over time. Clive Wearing's viral infection 
apparently destroyed hippocampal regions on both 
sides of the brain, but also some frontal lobe areas. 
Wearing may also have suffered brain damage that we 
simply do not know about. In the case of HM, how¬ 
ever, because his lesion was carefully performed by a 



312 


9. LEARNING AND MEMORY 


Functional framework for learning and memory 


Attention 


Vision 



Working memory 


Central 

executive 


Conscious 



. 

Att€ 


Attention 



Response 

output 



Explicit Implicit 

Learning Learning 

and retrieval and retrieval 



Major losses 
in amnesia: 


Perceptual 

Episodic (event) 

Semantic (fact) 

Implicit 

Habits and 

Primed biases 

memories 

memory 

memory 

memories 

motor skills 

and goals 


Long-term memory 


FIGURE 9.6 Functional framework: the typical loss in organic amnesia. Amnesia due to bilateral damage to 
MTL is highly specific. It impairs recollection of episodic (autobiographical) memories and blocks episodic learn¬ 
ing. It also makes it impossible to learn new facts and concepts (semantic learning). However, amnesics like HM 
and Clive Wearing can carry on a normal-seeming conversation because they can still retrieve semantic informa¬ 
tion that was learned before the injury. Implicit learning and retrieval also continue. Since the MTL and neocortex 
work together to enable episodic learning, damage to MTL on both sides of the brain seems to explain these spe¬ 
cific deficits. (Notice that the terms 'explicit' and 'conscious' are essentially equivalent; both are indexed by accu¬ 
rate report, as discussed in Chapter 8. Similarly, 'implicit' and 'unconscious' are equivalent for our purposes). 




FIGURE 9.7 Bilateral hippocampal damage in the classic patient HM. The 
lesions in HM's temporal lobes are shown in the coronal brain scan in the left panel. 
On the right is a normal scan for comparison, with the white arrows pointing to the 
medial temporal lobes. Those regions are missing in HM. The cutout brain image on 
the left will help you to understand the location of the coronal sections. Source : Left: 
Aminoff and Daroff, 2003; right : Corkin et al., 1997. 
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surgeon, we know that both sides of the medial tem¬ 
poral lobe (MTL) were removed as accurately as was 
possible at the time. The extent of HM's brain damage 
and functional deficits have been verified with great 
care, in more than 100 published articles. This has 
made HM one of the most important patients in the 
history of brain science (Box 9.1). 


2.2 A summary of amnesia 

HM represents the features of amnesia in a very 
'pure' form. More generally, amnesia is any loss of 
memory for personal experiences and other infor¬ 
mation, despite otherwise normal cognitive func¬ 
tions. The cause can be organic, including infection. 




FIGURE 9.8 The medial temporal lobes and HM's lesions, seen from below, (a) The orientation of the head and brain; (b) the 
bottom of the MTL, with major subregions for memory labeled. Notice that the rhinal (smell) cortices indicate the ancient origin of 
this region. In all figures you can see the two olfactory bulbs pointing upward, an important landmark for orientation, (c) The sur¬ 
gical lesion in HM's brain. The surgeon at the time was unaware of the importance of this region for memory Source : (b) Buckley 
and Gaffan, 2006; (c) Morris Moscovitch, personal communication. 


BOX 9.1 The case of HM 


The cognitive neuroscience of memory arguably began 
with Herbert Scoville and Brenda Milner's (1957) report 
of a memory disorder in HM after bilateral removal of 
his medial temporal lobes to control severe epileptic sei¬ 
zures. As a result of a head injury from a bicycle collision 
when he was a young boy, HM was beset with epileptic 
fits that increased in frequency and severity into his late 
20s. As a treatment of last resort, Scoville performed an 
operation in which he removed tissue in and around the 
hippocampus on both sides of HM's brain (see Figure 
9.8). While the surgery reduced HM's seizures, it had a 
profound and unexpected impact on his memory. This 


result was unknown at the time, and Scoville would 
undoubtedly have changed the procedure had he known 
about this harmful result. The report of HM's case by 
Scoville and Brenda Milner (1957) was the first to dem¬ 
onstrate directly the importance of the hippocampus and 
surrounding structures for memory. 

As a result of the operation, HM could not remember 
any of the events of his life thereafter - the people he 
met, the things he did, events taking place in the world 
around him. As an adult, he could not keep track of his 
age, and could no longer recognize himself in the mirror 
because he was unfamiliar with his changed appearance. 


(Continued) 
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BOX 9.1 (Continued) 


In addition to this anterograde (post-damage) memory 
deficit, HM also could not remember events or experi¬ 
ences from the years immediately before the surgery, a 
retrograde amnesia. While his episodic (autobiographical) 
memory loss was acute, other cognitive functions seemed 
to be intact. He could reason, solve problems, and carry 
on a normal conversation. His intelligence was normal, 
and he retained his language abilities. 

HM had intact short-term memory. He performed like 
healthy controls on tests of working memory, such as the 
digit span task. HM had been under the care of doctors from 
a young age, and his intellectual abilities before surgery were 
well documented. The specific locus of damaged tissue was 
both limited and well characterized. In most amnesias, the 
damage is more widespread and difficult to identify. HM 
had been tested and imaged a number of times since his 
surgery, giving a very complete picture of his condition. 


As you can tell from Figures 9.1 and 9.2, HM had an 
apparently intact neocortex - the outer structures in 
brain scan. Like Clive Wearing, HM could carry on a 
normal conversation. He could discuss the immediate 
present, using his general knowledge of the world. He 
was conscious, had normal voluntary control over his 
actions, and appeared to be emotionally well adjusted. It 
was only when his episodic memory was tested that he 
revealed that he simply could not remember the past, or 
learn new memories for the future. 

It is useful for you to review some important regions 
of the cortex (Figures 9.3, 9.9;). You should recall the loca¬ 
tion of the prefrontal lobes particularly, in front of the 
motor and premotor cortex. All of the neocortex is impor¬ 
tant for memory but the prefrontal lobes may be espe¬ 
cially important. 


stroke, tumor, drugs, oxygen deprivation, epilepsy, 
and degenerative diseases, such as Alzheimer's dis¬ 
ease. Amnesia can also be psychogenic , resulting from 
trauma or suggestion (Nilsson and Markowitsch, 
1999). 

As we have seen, organic amnesia is caused by 
bilateral damage to the medial temporal lobes, which 
includes the hippocampal formation. It generally reveals: 

1 Impaired memory but preserved perception , cognition , 
intelligence , and action. Amnesic people perform 
normally on standard tests of intelligence, but are 
impaired on standard tests of memory. They can play 
chess, solve crossword and jigsaw puzzles, compre¬ 
hend complex instructions, and reason logically. 

2 Impaired long-term but not working memory. Amnesic 
people have a normal digit span. They are impaired, 
however, if they are distracted. The same holds for 
words, stories, visual patterns, faces, melodies, 
smells, and touch. 

3 Impaired recent but not remote memories. Memory 
loss is most noticeable for events learned after 
the onset of the disorder, as well as in the period 
immediately preceding it, but not for information 
acquired years before. That is, amnesia victims 
have an anterograde amnesia that extends into the 
future but a limited retrograde amnesia. The length 
and severity of retrograde amnesia varies. 

4 Impaired explicit but not implicit memory. Anterograde 
(post-injury) memory loss applies only to informa¬ 
tion that can be remembered consciously or explic¬ 
itly. Learning, retention, and retrieval of memory 
without awareness or implicitly is normal. 


2.3 Spared functions in amnesia: implicit and 
procedural memory 

As mentioned above, implicit memory is commonly 
assessed by priming tasks. Perceptual priming is 
mediated by sensory cortex, and conceptual priming 
is believed to involve both the temporal and prefron¬ 
tal regions. Amnesics do not lose their capacity to per¬ 
form well on priming tasks, such as word-fragment 
completion. For example, subjects may study a list of 
words (such as metal) and are tested with fragments 
of the words they studied, to see if they can complete 

them ( met _). The study phase increases the speed of 

completion. On such tasks, amnesic patients can per¬ 
form as well as normals. 

Functional neuroimaging studies confirm the per¬ 
ceptual locus of perceptual priming. Unlike tests of 
explicit memory, which are associated with increased 
activation during retrieval in regions that support 
memory performance, such as the MTL and prefrontal 
cortex, perceptual priming is associated with decreased 
activation on repeated presentations in regions believed 
to mediate perceptual representations. Thus, repeated 
presentation of faces and words leads to decreases in 
activation in inferior temporal and extrastriate cortex 
which mediate face and word perception (Wigg and 
Martin, 1998; Schacter et ah, 2004). 


2.3.1 Conceptual priming 

In conceptual priming the relationship between study 
and test items is meaning-based. Conceptual tasks 
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include word association (Tell me the first word that 
comes to mind for Elephant'), category exemplar 
generation ('Provide examples of animals'), and gen¬ 
eral knowledge ('What animal is used to carry heavy 
loads in India?') (Moscovitch et al., 1993; Roediger 
and McDermott, 1993). Conceptual priming occurs if 
studied words (e.g. elephant) are retrieved more fre¬ 
quently than unstudied ones. Because conceptual 
priming depends on meaning, a change in the physi¬ 
cal form of stimuli has little influence on conceptual 
priming. 

Conceptual priming is impaired in people with 
damage to regions of cortex mediating semantics. 
Thus, patients with semantic dementia whose degen¬ 
eration affects lateral and anterior temporal lobes 
show an inability to recognize repeated objects which 
share a common meaning, for example, two different- 
looking telephones. But they have no difficulty rec¬ 
ognizing the object if it is repeated in identical form 
(Graham et al., 2000). 

Likewise, patients with Alzheimer's disease show 
preserved perceptual priming but impaired conceptual 
priming. Functional neuroimaging studies of concep¬ 
tual priming implicate semantic processing regions, 
such as prefrontal and lateral temporal areas. As in 
tests of perceptual priming, tests of conceptual prim¬ 
ing lead to decreases in activation in those regions 
(Buckner et al, 1998; Schacter et al., 2004). Asking 
people repeatedly to generate examples of 'animals' 
results in reduced activation in the same regions. 

Numerous studies have shown that priming on 
a variety of tests is normal in amnesic patients. This 
applies to most tests of perceptual and conceptual 
priming, indicating that the MTL does not contribute 
to them. 

2 . 3.2 Spared procedural memory in amnesia 

One of the earliest demonstrations of preserved 
memory in amnesia was on tests of learning percep¬ 
tual motor skills called procedural memory. Corkin 
(1965) and Milner (1965) showed that HM was able 
to learn and retain a pursuit-rotor task, keeping a 
pointer on a moving target. HM showed improve¬ 
ment on these tasks even months later, though he 
could not recall doing it even minutes afterward if he 
was distracted. These findings have been repeated in 
other cases. 

Procedural memory depends on perceptual-motor 
regions, like the basal ganglia, which interact with 
the neocortex, both the posterior and frontal (see 



FIGURE 9.9 Neocortex: motor, premotor, and prefrontal 
regions. In this lateral view of the cortex, the motor strip is tinted 
purple, and just in front of it, the premotor area encodes the cog¬ 
nitive intention to move specific limbs. The true prefrontal cortex 
is in front of these motor regions (light green). The light blue area 
of the prefrontal lobe is sometimes called the orbitofrontal cortex, 
because it is located just above the orbital hollows of the two eyes. 
You are encouraged to review the other visible structures, including 
the major lobes, the Sylvian fissure, and the central sulcus, which 
divides the posterior half of cortex from the frontal half. Source : 
Drawn by Shawn Fu. 


Chapter 5). Patients with impaired basal ganglia due 
to Parkinson's or Huntington's disease show little or 
no improvement after practicing sensorimotor tasks 
(Kaszniak, 1990; Gabrieli et al., 1994). 

In the serial reaction time task (SRT), dots are pre¬ 
sented on a computer screen in one of four locations, and 
participants are instructed to press a key corresponding 
to the location of the dots they see (Willingham et al., 
1989). Some of the sequences are repeated while oth¬ 
ers change randomly. Reaction time to the repeated 
sequences becomes faster with repetition, but remain 
the same for the random sequences, even though partic¬ 
ipants cannot consciously distinguish one from the other 
(Willingham, 1998). Amnesic patients perform normally 
on the implicit SRT task but poorly on the explicit ver¬ 
sion (Nissen and Bullemer, 1987; Reber and Squire, 
1998). Again, patients with basal ganglia disorders like 
Parkinson's disease perform poorly on both SRT tasks 
(Knopman and Nissen, 1987; Vakil et al., 2000). 

Functional neuroimaging studies also show 
that learning on the implicit SRT task is associated 
with activity in the basal ganglia but not with MTL 
activity. 
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2 A Spared implicit learning 

Academic learning is often explicit: professors point 
out the things to be learned, and students try their best 
to memorize them. But most ordinary human learning 
is probably implicit. A hunter may teach young people 
how to track an animal or how to kill and skin it. Most 
of the time such practical activities can be taught more 
easily by modeling than by explicit labeling. Many of 
the subtleties of hunting and gathering may not even 
have names. Experimentally, subjects who are given a 
set of stimuli generated by a simple set of rules uncon¬ 
sciously infer the underlying regularities. Social habits 
are probably learned mostly implicitly. Children learn¬ 
ing language surely don't label the words they hear as 
nouns or verbs. Rather, they pay attention to speech 
sounds and the underlying regularities are learned 
implicitly. We rarely become conscious of abstract pat¬ 
terns - the regularities of grammar, the harmonic pro¬ 
gressions of a symphony, or the delicate brushwork 
of a work of art. Most knowledge is tacit knowledge; 
most learning is implicit. 

Knowlton et al. (1994, 1996) used a probabilis¬ 
tic classification task to study implicit learning. 


Participants were shown sets of four cards that pre¬ 
dicted the 'weather' with different probabilities 
(Figure 9.10). After each set of cards, participants were 
asked whether it predicted 'rain' or 'sunshine'. Some 
sets predicted 'rain' 20 percent of the time, and others 
80 percent of the time. Learning this association takes 
about 50 trials in normal people, but it takes consider¬ 
ably longer for subjects to realize explicitly which card 
sequence predicted which kind of weather. That is, the 
weather prediction is learned implicitly before it can be 
stated explicitly. 

Amnesia patients performed as well as controls 
in the early trials, the implicit association, but not 
in the later trials when performance was based on 
explicit, declarative memory. However, patients with 
Parkinson's disease, a basal ganglia dysfunction, per¬ 
formed poorly during the early trials, but caught up 
during the later ones. Brain imaging showed activity 
in the basal ganglia throughout the task but a shift in 
MTL activity as the task progressed (Poldrack et al., 
1998). 

In summary, amnesia due to bilateral damage 
to MTL seems to be primarily a disorder of episodic 



In this game you are a weather forecaster. 
You will learn how to predict rain or shine using 
a deck of four cards. 


A 




FIGURE 9.10 An implicit "weather prediction' learning task. Knowlton et al. (1994) devised this 'weather pre¬ 
diction' task for four-card sequences. Participants learned to predict the probabilistic outcomes implicitly before 
they were able to state the pattern explicitly. Amnesic patients performed well during the implicit early part of the 
learning process, but did not learn to state the association between the cards and the 'weather' explicitly. Source : 
Knowlton et al., 2003. 
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memory, resulting from impaired transfer of information 
from working memory into long-term memory. 
Because memories acquired long before the onset of 
amnesia are relatively spared, it is believed that the 


hippocampus and related structures in the medial 
temporal lobe are needed only temporarily to hold 
information in memory until they are consolidated 
elsewhere in the brain, presumably in neocortex. 


FRONTIERS OF COGNITIVE NEUROSCIENCE 


Episodic memory 



FIGURE 9.11 Charan Ranganath, PhD, University of 
California, Davis, CA, USA. 

The great psychologist Endel Tulving (Tulving, 1985) 
described episodic memory (the ability to remember a past 
event) as "mental time travel." It took neuroscientists a 
couple of decades to catch up with Endel, but we now 
understand that episodic memory allows us to reexperi¬ 
ence the past, stay oriented in the present, and plan for 
the future (Schacter et al., 2007). Tragically, memory is 
significantly affected by numerous neurological (e.g. 
Alzheimer's disease, fronto-temporal dementia, epilepsy, 
traumatic brain injury, etc.) and psychiatric (e.g. schizo¬ 
phrenia, depression, post-traumatic stress disorder, etc.) 
conditions, and the consequences can be debilitating. 
Accordingly, understanding the functional organization 
of memory processes and their neural substrates is of 
supreme importance to both scientists and society. 

Until recently, much of what we knew about the 
neural basis of memory came primarily from animal 
models and comparison with patients with human 
amnesia. This research yielded a great deal of informa¬ 
tion about the types of memory that might be supported 


by particular neural systems, but what was missing 
was an understanding of how these systems work in 
the human brain. Fortunately, when I was in graduate 
school, researchers began to use brain imaging tech¬ 
niques to identify the neural correlates of memory proc¬ 
esses. These techniques have allowed us to develop and 
test models to explain not only imaging data, but also 
results from studies at the systems and molecular levels 
in animals and studies on the functional nature of mem¬ 
ory processes in humans. 

Of course, the $64,000 question is, have we learned 
anything from all those pretty pictures? I certainly think 
so. Since Brenda Milner's groundbreaking work on 
amnesic patient Henry Molaison ("HM"), we've known 
that the medial temporal lobe (MTL) region is criti¬ 
cal for episodic memory. Imaging studies have added 
to this picture by demonstrating that different areas of 
the MTL support memory in different ways (see Figure 
9.12). For instance, perirhinal cortex activation is linked 
to recognition of items based on familiarity, whereas 
activity in the hippocampus and parahippocampal cor¬ 
tex is linked to recollection of relationships between 
items and the context in which they were encountered 
(Diana et ah, 2007). This pattern of results converges 
remarkably with studies of focal MTL lesions in rats 
and monkeys, suggesting parallels in MTL function that 
are surprisingly conserved across species (Eichenbaum 
et ah, 2007). Imaging studies have also shown that epi¬ 
sodic memory is supported by a cortical network that 
extends far beyond the MTL. For instance, imaging 
findings have highlighted the fact that the prefrontal 
(Ranganath and Blumenfeld, 2008) and posterior pari¬ 
etal cortices (Cabeza et ah, 2008) are routinely recruited 
in the service of memory encoding and retrieval, and 
we are just beginning to understand the roles of these 
regions (Figure 9.12). 

What I've just described only hints at the potential for 
future developments in this area. Recent breakthroughs 
in high resolution imaging, coupled with analysis tech¬ 
niques to detect and classify spatial patterns in imaging 
data, will lead to significant changes and refinements in 
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FIGURE 9.12 Prefrontal and medial temporal lobe regions 
that contribute to episodic memory processing, (a) Relative loca¬ 
tions of the dorsolateral prefrontal cortex ('DLPFC'; light blue), 
ventrolateral prefrontal cortex ('VLPFC'; peach), perirhinal cortex 
(blue), parahippocampal cortex (green), and hippocampus (red) 
are shown on a rendering of a brain with a cutaway to reveal 
the medial temporal lobes, (b) A diagram illustrating our cur¬ 
rent model of how lateral prefrontal and medial temporal regions 
may contribute to episodic memory. The anatomical connections 
between each region are illustrated with black lines and proposed 
roles of each region are shown in italic text. For simplicity, the dia¬ 
gram presents only the most significant anatomical connections 
between these regions and omits other anatomically connected 
regions that also may play a significant role in episodic memory 
formation or retrieval. Source: Ranganath, (in press). 


our theories of the neurocognitive processes that support 
our amazing capability for mental time travel. 
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3*0 MEMORIES ARE MADE OF THIS 

Traditionally, a memory is considered to be a stable 
record of an event, which can be recalled accurately 
in the same form it was learned. In this common- 
sense view, memories can be retrieved, examined, 
and played back like a high fidelity music record¬ 
ing. Memories can also be forgotten without affecting 
other cognitive systems. 

There are reasons to question this common sense 
idea. One is that real memories of past events are 
rarely accurate. The process view considers memory 


to be a product of a dynamic process, a reconstruc¬ 
tion of the past influenced by past and current con¬ 
ditions, anticipations of future outcomes, and other 
cognitive processes. In the process view, memory is 
based on stored information, but is not equivalent to 
it. It is dynamic and mutable, and interacts with other 
processes. Thus, two people experiencing the same 
event may have different memories of it. It is not sim¬ 
ply that one person is right and the other wrong but 
that in retrieving the memory each person's outlook, 
knowledge, motivation, and retentive abilities may 
alter what is retrieved. 
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Everyone's memory changes with time. We forget 
most of what has happened within minutes or hours, 
and what remains is commonly reorganized and dis¬ 
torted by other knowledge or biases. We would not 
want computer files or books to be that way. We do not 
want files to decay over time or to leak into neighbor¬ 
ing files. Computers and libraries are designed to keep 
everything as distinct and stable as possible. Yet normal 
memories do fade, and are often confused with others. 

Try to reconstruct in as much detail as possi¬ 
ble all the things you did two weekends ago, in the 
exact order in which you did them. To do that, most 
of us have to search for cues to determine exactly 
what we did. Having found a cue, there is a process 
of reconstruction, especially in trying to figure out 
the sequence of events. Did I meet my friend before 
I spoke to my parents or afterward? Did I go shop¬ 
ping, and what was the order of the stores I visited? In 
each store, in what order did I look at the merchandise 
and buy it? You can try this with a recent movie, and 
then see how accurate your memory is by checking it 
against a copy of the movie. 

To answer these questions you must draw on a body 
of knowledge and inference that is unlike anything 
that is needed when you enter a file name to access 
a computer file or use a call number to find a library 
book. You may confuse what you did two weekends 
ago with what happened another time. As we will 
see, some patients with brain damage have a disorder 
called confabulation, in which they make up false mem¬ 
ories without any intention of lying, and without any 
awareness that their memories are incorrect. 

Memories influence how other memories are 
formed and retrieved. They also shape our actions 
even when we are not conscious that they do so. Our 
memories and dispositions influence our thoughts and 
actions and, in turn, are influenced by them. In short, 
memory is needed to carry on the affairs of daily life 
and to plan for the future, and is, in turn influenced 
by the past, whether conscious or not, and by our 
thoughts about the future. 


3*1 Electrically evoked autobiographical 
memories 

For some 50 years, brain surgeons have reported that 
awake patients report vivid, specific conscious recol¬ 
lections during temporal lobe stimulation. Penfield 
and Roberts (1959) were among the first to use pin¬ 
point electrical stimulation of cortex to map out func¬ 
tional regions. Their aim was to remove 'epileptic foci' 


in the cortex, regions of scarred tissue that can trigger 
the massive electrical storm of major seizures. In order 
to locate epileptic foci, and to avoid harming function¬ 
ally important areas, the cortex of awake patients was 
mapped using low-voltage electrical stimulation in 
specific cortical locations. 

This is only possible because the cortex itself has no 
pain receptors. As long as local anesthetic is used to 
block pain from the scalp incision, conscious patients 
can talk about their experiences without harm. Open- 
brain surgery provides a unique source of evidence 
for cognitive neuroscience. 

Stimulation of the temporal lobe sometimes results 
in an unexpected flood of conscious memories. For 
example, a recent patient during brain stimulation 
gave the following reports of his experiences (Moriarty 
et al ., 2001): 

1 At four electrode locations shown in Figure 9.13, 
re-experiencing Flinstones cartoons from childhood 

2 At four different electrode locations, hearing the 
rock band Pink Floyd 

3 At two other locations, a baseball announcer 

4 At four more locations, an unknown female voice. 

Notice that the open circles in the figure show elec¬ 
trode locations that led to no experiential memories at 
all. Some of the electrode sites are close to the audi¬ 
tory cortex of the upper temporal lobe, but some are 
not. And these auditory regions are not known to pro¬ 
vide such rich memories of experiences that patients 
report having heard at specific points in their lives. 
However, this particular patient had a surgical lesion 
in the medial temporal lobe, and we must be careful 
about generalization to other cases. 

The patient in this case is not told which electrode 
is currently stimulating the temporal lobe, so that it is 
not possible to fake this pattern of results. Scientists 
have been skeptical about these reports, since stimu¬ 
lation in one place might activate other regions as 
well. Epileptics have atypical brains, since cortex com¬ 
monly changes in response to disorders. However, it 
is extremely unlikely that so many different patients 
would report a flow of conscious memories over fifty 
years of stimulation of the temporal lobe. After numer¬ 
ous animal experiments, careful studies of amnesic 
patients and brain imaging of memory tasks, there is 
little doubt today that there is something about the 
temporal lobe that is specific to long-term episodic 
memory. One reasonable hypothesis is that temporal 
lobe stimulation somehow activates specific memories 
by way of the MTL. 
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Figure 9.14 shows a possible explanation of this 
phenomenon. What we are seeing in the neurosurgical 
patient is an established memory pattern, which is re¬ 
evoked by direct electrical stimulation to the temporal 


lobe. Thus, the flow of information is from the neo¬ 
cortex (temporal) to the hippocampal system (MTL), 
which causes the hippocampal system to resonate with 
the original neocortical memory traces to produce the 
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13-14 Pink Floyd 

8- 9 Pink Floyd 
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7-8 Female voice singing 
12-13 Female voice singing 


FIGURE 9.13 Autobiographical memories evoked by temporal lobe stimulation. Spontaneous reports of 
memory experiences by electrical brain stimulation in a patient with a surgical lesion in the left medial temporal 
lobe. Notice the electrode grid that was placed on the cortex as shown. Electrodes were places 1 cm apart. Different 
electrodes consistently evoked different memory episodes. Spontaneous reports of this kind are not unusual with 
temporal lobe stimulation, but are not routinely reported as a result of other locations of cortical stimulation. Source : 
Moriarty et ah, 2001. 
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FIGURE 9.14 The hippocampal system (MTL) and neocortex in learning and recall. A neural net model 
of MTL (the hippocampal system) in interaction with neocortex. During the encoding or learning process (a), 
information from cortex is transferred to the hippocampal system. During recall (b) and (c), a neocortical event 
serves to evoke an overlapping pattern of neural activation in the MTL (the blue dots). The hippocampal sys¬ 
tem responds by activating neocortical regions that provide the experience of recall of some part of the original 
event. Source : Gluck et ah, 2003. 
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original episodic experience, or something very much 
like it. However, this still leaves many questions unan¬ 
swered, such as, what is the relationship between the 
electrical stimulus and the normal process of episodic 
memory retrieval? Why are such extremely specific 
memories evoked under these circumstances, among 
the many millions of episodes the average person has 
experienced? Is there perhaps a relationship between 
the electrical stimulus and the oscillatory EEG phe¬ 
nomena that are observed in connection with retrieval, 
like the theta rhythms that seem to coordinate MTL- 
neocortical retrieval (see below)? Episodic memories 
reported by patients in neurosurgery after electrical 
stimulation are fascinating, and appear to be robust 
and reliable phenomena. But we do not have a good 
specific explanation as yet. 

3*2 Long-term potentiation and long-term 
depression: excitatory and inhibitory 
memory traces 

Most synapses in cortex are excitatory, using the neu¬ 
rotransmitter glutamate (see Chapter 3). A very large 


minority use inhibitory neurotransmitters like GABA 
(gamma-aminobutyric acid). To encode long-term 
memory traces in changed synaptic efficiency, these 
excitatory and inhibitory connections must somehow 
be made more permanent. These two processes are 
believed to occur in what is called long-term poten¬ 
tiation (LTP) for excitatory synapses and long-term 
depression (LTD) for inhibitory ones. These events, 
which have been observed in specific regions, are sim¬ 
ply an increase and a decrease in the firing probability 
of a postsynaptic potential given a presynaptic spike. 

LTP has been observed within the hippocampus 
itself, using single-cell recording in one of the neuro¬ 
nal layers of the hippocampus (Figure 9.15). Single¬ 
cell recording has been extensively done in animals, 
but there are cases of such recordings in human epi¬ 
leptic patients as well (Kreiman et al., 2002). 

While we can observe LTP and LTD in specific loca¬ 
tions like the hippocampus, the standard hypothesis 
about long-term memory involves billions of synapses 
in cortex and its satellites, amounting to literally tril¬ 
lions of synapses. We have no way of taking a census 
of all of the synapses in this system, or even a substan¬ 
tial fraction of them, at this time. Rather, we have a 
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FIGURE 9.15 Single-cell recording in hippocampus. A schematic of single cell recording and 
stimulation either inside of hippocampal neurons or outside. An electrode placed outside of a neuron 
can pick up electrical field potentials, which are similar to EEG but much more localized. Electrical 
field potentials often reflect the activity of small populations of neurons, as opposed to the axonal or 
dendritic potential of a single neuron. Notice that a very similar technique can be used to stimulate 
single cells, or small sets of cells, in the hippocampus. Source : Squire et al., 2003. 
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number of studies showing increased LTP and LTD in 
experiments like the one described in Figure 9.16, sup¬ 
plemented by studies of brain damage and of popu¬ 
lation activity among billions of neurons as measured 
by EEG, ERP, fMRI, and so on. In addition, we have 
evidence from stimulation studies, like temporal lobe 
stimulation of awake patients during neurosurgery, 
and transcranial magnetic stimulation (TMS) in nor¬ 
mal subjects. What we know about memory is there¬ 
fore an inferential picture, in which many hundreds of 
studies have been performed. But we cannot yet come 
close to observing large numbers of changed synaptic 
connectivities directly at the submicroscopic level. 

With that caution in mind, there is good agree¬ 
ment today that the evidence is consistent with these 
propositions: 

1 Episodic input is initially represented via neocortex. 


2 It is integrated for memory purposes in the MTL, 
containing the hippocampi and related structures, 
and perhaps also the thalamus and surrounding 
regions. 

3 Consolidation: MTL and related regions then bind 
and integrate a number of neocortical regions, a 
process that transforms temporary synaptic con¬ 
nectivities into longer-lasting memory traces in 
both MTL and neocortex. The main mechanism for 
such changes is believed to be LTP and LTD. 

We focus on the last point next. 

3*3 Consolidation: from temporary to 
permanent storage 

We can now add the final steps. Chapter 2 suggested 

a widely accepted hypothesis about the relationship 
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FIGURE 9.16 Long-term potentiation in the hippocampus. Memory traces are believed to be encoded in 
changed synaptic efficiency among billions of neurons in the neocortex and MTL. These are called long-term poten¬ 
tiation (LTP), corresponding to a permanent increase in excitatory transmission, and long-term depression (LTD), 
a permanent increase in inhibitory neurotransmission. Direct evidence for LTP has been obtained in hippocampal 
single cell recordings, as shown above. The lower half of the diagram shows three cases of changed EPSP (excita¬ 
tory postsynaptic potentials) after strong electrical stimulation presynaptically (called tetanus). Notice that after 90 
minutes, the EPSP dips more deeply (remember that a more negative potential means more electrical spiking activ¬ 
ity). The graph above shows a long-lasting change in the conductivity of the synapse, as measured by the slope of 
the EPSPs for 2 hours after the strong electrical stimulus (tetanus). Source : Byrne in Squire et al., 2003. 
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between immediate memory and long-term mem¬ 
ory called the consolidation hypothesis. Consolidation 
is generally defined as a progressive stabilization 
of long-term memory traces so that they are rela¬ 
tively resistant to decay or disruption. It is this pro¬ 
cess which is absent or severely disrupted in amnesic 
patients and accounts for their poor ability to transfer 
information from short- to long-term memory. 

Figure 9.17 shows one version of learning with 
consolidation, in which input into the neocortex and 
the hippocampal regions (MTL) evoke an active state, 
with neuronal processes making new synaptic con¬ 
nections. As mentioned above, immediate memory is 
encoded in improved synaptic connectivity between 
billions of neurons in the neocortex. Normal sleep, 
especially the slow-wave stage, is important to turn 
these temporary connectivities into long-lasting mem¬ 
ory traces. 

However, more permanent memories are believed 
to require protein synthesis - such as the growth of 
dendritic spikes, tiny stalks that grow on top of axons 
and dendrites, bearing new synaptic connections with 
neighboring neurons. 

The idea that new learning takes some time to 'fix' 
is quite old. In 1904, Burnham wrote that: 

The fixing of an impression depends on a physiological 
process. It takes time for an impression to become so 
fixed that it can be reproduced after a long interval; for 
it to become part of the permanent store of memory con¬ 
siderable time may be necessary. This we may suppose 


is not merely a process of making a permanent impres¬ 
sion upon the nerve cells, but also a process of associa¬ 
tion, of organization of the new impressions with the 
old ones (quoted by Moscovitch). 

Figure 9.18 shows these two kinds of consolidation 
conceptually. The LTP-LTD process discussed above 
involves cellular consolidation, a local change of con¬ 
nective efficiency in trillions of synapses. However, 
there is also believed to be systems consolidation , in 
which large-scale reorganization of memories may 
occur. There is considerable evidence that different 
sleep stages may have different effects upon this sys¬ 
tems consolidation process. Notice that both types of 
consolidations are thought to involve an active dia¬ 
logue between the MTL (hippocampus) and neocortex. 


3.4 Rapid consolidation: synaptic mechanisms, 
gene transcription, and protein synthesis 

Rapid or synaptic consolidation is accomplished within 
the first minutes to hours after learning occurs. Weiler 
and colleagues (Weiler et al., 1995) showed that it corre¬ 
lates with morphological changes in the synapse itself. 
Stimulus presentation initiates a cascade of neurochem¬ 
ical events at the synaptic membrane and within the 
cell which increase the synaptic strength or efficiency 
with which neurons that form the memory trace can 
communicate with one another. The first of these proc¬ 
esses involves local, transient molecular modifications 


LEARNING BINDING 
Event Event 

/i\ 

HUE HHH 

^ ^ j' l/ l 


CONSOLIDATION 


REMEMBERING 

Event 


s/i\ 

□EH 

Iff 



©®© 


MTL 


1 r -“ - —> 

B 

4 _ J 

( -^ 

ABC 

_ A _ jk 

XTZ 

f i 

MTL 

^ _ 4 

( ^ w r X 

MTL 

*_ J 


FIGURE 9.17 The steps of learning, binding, consolidation, and remembering. In this summary. 
Step 1 is the learning of an event, consisting of three elements. A, B, and C. It is initially encoded by 
neocortex (such as the visual cortex) and sent to MTL. In Step 2, MTL and neocortex resonate with each 
other to begin establishing the memory trace. In Step 3, the stimulus event is no longer available, and 
the MTL-neocortical resonance is now independent of external support. Step 4 shows how consolida¬ 
tion leads to permanent, separate memory traces (synaptic changes) in both the MTL and neocortex, 
which now exist separately from each other, while other input is being processed. In Step 5, element B of 
the original event (A-B-C) is presented as a reminder or recall cue. In Step 6, the memory traces of A-B- 
C are activated by resonating activity between MTL and neocortex. At this point, the episodic memory 
has been retrieved in the absence of the original stimulus. Source: Morris Moscovitch, modified with 
permission. 





















324 


9. LEARNING AND MEMORY 



Hippocampus Neocortex 



Active Inactive Active Inactive 

state state state state 




Short-term memory (STM) 

• Lasts for seconds to hours 

• Labile 1 (sensitive to disruption) 

• Does not require new RNA or 
protein synthesis 


Long-term memory (LTM) 

•Lasts for days to weeks 

•Consolidated (insensitive 
to disruption) 

• Does require new RNA 
or protein synthesis 



Cellular ^ 
reconsol relation 

-► 

Systems 

{c) reconsolidation 


(a) 



Active state (AS) 

• Lasts for seconds to hours 


Inactive state (IS) 

• Lasts for days to weeks 


• 1 La bile 1 (sensitive to disruption) 

(Does not require new 
RNA or protein synthesis) 


* Inactive (insensitive 
to disruption) 

(Does require new RNA 
or protein synthesis) 



M 

FIGURE 9.18 Reconsolidation turns active neuronal connections into lasting ones. Two kinds of consolidation are believed 
to exist, cellular and systems consolidation. Both are evoked by activation of MTL and neocortex. This diagram emphasizes the 
degree to which MTL (also called the hippocampal complex) and neocortex establish active cell assemblies corresponding to the 
learned input, in which neurons resonate with each other until more permanent connections are established. Source : Nader, 2003. 


that lead to an increase in neurotransmitter release at 
the affected synapse. If the stimulus is intense enough 
and/or repeated, additional processes are activated. 
These involve gene transcription and protein formation 
that lead to long-lasting cellular changes, including the 
creation of new synapses, that support the formation 
and maintenance of long-term memory. These proc¬ 
esses may last from hours to days (Figure 9.19) (Lees 
et ah, 2000; McGaugh, 2000; Dudai, 2004). 

Though we are well on our way to understanding 
the basic cellular and molecular mechanisms of syn¬ 
aptic consolidation, we are far from understanding 
prolonged or system consolidation, which is being 
debated heatedly in the literature. 

3*5 System consolidation: interaction between 
the medial temporal lobes and neocortex 

System consolidation can take much longer to com¬ 
plete and may range from days to years or decades. 


Patients with MTL lesions show a retrograde memory 
loss that is temporally graded, so that recent memory 
loss (before the amnesia) is greater than earlier mem¬ 
ory loss. This temporal gradient is restricted to explicit 
memory, leaving implicit memory intact and stable 
over time (Scoville and Milner, 1957). 

These observations suggest that the MTL forms a 
temporary memory trace needed for explicit memo¬ 
ries until they are consolidated elsewhere in the brain, 
presumably in the neocortex (Squire, 1992; Squire and 
Alvarez, 1995). This standard model of consolidation 
makes no distinction between various types of explicit 
memory. For instance, it predicts a similar pattern for 
episodic and semantic memory. 

Nadel and Moscovitch concluded, contrary to the 
standard consolidation model, that MTL is needed 
to represent even old episodic memories for as long 
as the memory exists (Nadel and Moscovitch, 1997, 
1998; Moscovitch and Nadel, 1998; Nadel et al, 2000). 
Neocortex, on the other hand, is sufficient to represent 
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FIGURE 9.19 The time course of consolidation. McGaugh 
(2000) suggests that there are three overlapping time courses for 
consolidation. The fastest is referred to as "short-term memory", 
from seconds to hours. Long-term memory consolidation takes 
place over hours to months. Finally, long-lasting memory is needed 
to account for certain facts, such as the retained long-term memory 
of early life events in amnesics, who do not have spared memory 
for some years before brain damage occurred, and none for the time 
afterward. Source : McGaugh, 2000. 

repeated experiences with words, objects, people, and 
environments. MTL may aid in the initial formation of 
these neocortical traces, but once formed they can exist 
on their own. Thus, unique autobiographical memo¬ 
ries are different from repeated memories, in that they 
continue to require MTL. Repeated experiences are 
proposed to create multiple traces, adding more traces 
each time the event is brought to mind. 

Neuroimaging studies provide evidence for this 
interpretation (Box 9.2). These studies found that the 
hippocampus is activated equally during retrieval of 
recent and remote autobiographical memories (Conway 
et ah, 1999; Ryan et ah, 2000; Gilboa et ah, 2004; for 
review see Maguire, 2000; Moscovitch et ah, 2005,2006). 
These questions continue to be debated at this time. 

4*0 VARIETIES OF MEMORY 

Memory is not unitary. Clive Wearing's memory 
for specific past events in his life is almost entirely 
destroyed, but he can maintain active knowledge of 
the immediate past for about 7 seconds (if he is not 
distracted). Clearly, some forms of memory have been 
targeted by his disorder and others not. This pattern is 
routine in amnesic patients, and suggests that different 
types of memory have different neural underpinnings. 
A standard view of the major long-term memory sys¬ 
tems is shown in Figure 9.20. However, the exact rela¬ 
tionship between memory types is still a matter of 
debate. For example, perceptual memory is classified 


BOX 9.2 Multiple trace theory versus traditional 
consolidation 


Nadel and Moscovitch (1997) proposed a multiple trace 
theory, suggesting that the hippocampal complex rap¬ 
idly encodes all information that becomes conscious. 
MTL binds the neocortical neurons that represent the 
conscious experience into a memory trace. MTL neu¬ 
rons act as a pointer, or index, to the neocortical ensem¬ 
ble of neurons that represent the experience (Teyler 
and DiScenna, 1986). A memory trace of an episode, 
therefore, consists of a bound ensemble of neocortical 
and MTL neurons. Formation of these traces is rela¬ 
tively rapid, lasting on the order of seconds or at most 
days (Moscovitch, 1995). 

In this model, there is no prolonged consolida¬ 
tion process that slowly strengthens the neocortical 
memory trace. Instead, each time an old memory is 
retrieved, a new hippocampally-mediated trace is cre¬ 
ated, so that old memories are represented by more 
traces than new ones, and therefore are less suscepti¬ 
ble to disruption. Because the memory trace is distrib¬ 
uted in the MTL, the extent and severity of retrograde 
amnesia is related to the amount and location of dam¬ 
age to the MTL. 

While each autobiographical memory trace is 
unique, the existence of many related traces facilitates 
retrieval. Episodic memories are integrated to form 
semantic memories. Thus, facts about people and 
events that are learned in specific episodes become 
separated from their sources. This process may give 
the appearance of classical consolidation, but the brain 
mechanism is different from the classical view. 


under non-declarative memory, but this is rather arbi¬ 
trary. Perceptual memory manifests in an improve¬ 
ment in sensory discrimination at the cortical level, 
often by reorganization of cortical receptive fields. The 
result is a change in conscious perception, as in learn¬ 
ing to identify clearly the sound of a guitar in music. 
This has much in common with episodic memory, 
which can also be largely perceptual and which is 
generally believed to be conscious. Nevertheless, the 
memory classification shown in Figure 9.20 is widely 
used, and is part of the vocabulary students of the 
field are expected to know. 

4*1 Episodic and semantic memory: 
‘remembering* versus ‘knowing* 

As Figure 9.20 shows, declarative memory can be 
divided into two types, episodic and semantic (Tulving, 
1972). Episodic memory refers to memories that have a 
specific source in time, space, and life circumstances. 
Episodic memories are often autobiographical in nature, 
in that we can travel mentally back in time to relive 
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FIGURE 9.20 A classification of mem¬ 
ory types. Schacter and Tulving proposed this 
classification of memory types. Declarative 
memories have been studied in great detail, 
and are believed to be explicit (conscious). 
Non-declarative memory types are said to be 
unconscious or implicit, but this claim is still 
debated. While the non-declarative memory 
types in this diagram undoubtedly have 
unconscious aspects, it is not yet clear that 
they can be learned without conscious input. 
Source : Squire, 2004. 
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FIGURE 9.21 Remembering: autobiographical episodes. 
Remembering involves an active reconstruction of the original (con¬ 
scious) episode. These conscious recollections seem to require hip¬ 
pocampal activity. Source : Morris Moscovitch, with permission. 
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FIGURE 9.22 Knowing: semantic information. Semantic 
memories are assessed by feelings of knowing, which can be very 
accurate. However, they do not require active reconstruction of 
the original episode, and can apparently be accomplished by neo¬ 
cortex without the aid of the hippocampal complex. Source : Morris 
Moscovitch, with permission. 


the experience. By contrast, semantic memories involve 
facts about the world, about ourselves, and about other 
knowledge that we share with a community. Semantic 
memories are independent of the spatial and temporal 
context in which they were acquired. A semantic mem¬ 
ory may refer to our knowledge of Paris as the capital of 
France, or Ottawa as the capital of Canada, or the knowl¬ 
edge that we attended a particular high school. By com¬ 
parison, episodic memory may refer to an event that we 


experienced in Paris, Ottawa, or high school (Figure 9.21). 
We often have an autobiographical source memory about 
a specific time, place, and set of circumstances when our 
episodic memory was acquired. The terms 'episodic' 
and 'autobiographical' memory are near synonyms. 

Episodic memories typically: 

1 have reference to oneself 

2 are organized around a specific time period 
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BOX 9.3 False feelings of familiarity: AKP, a person with deja vecu 


People can falsely believe they have known something 
before when they have not. AKP was an 80-year-old 
Polish immigrant to Britain, a retired engineer who had 
a master's degree in his field. He presented to his fam¬ 
ily doctor with memory problems, and complaining of 
frequent sensations of what his wife described as deja vu 
(actually, deja vecu, the feeling of having lived through 
the same experience before). The sensation of deja vecu 
was so strong that it influenced AKP's daily activities. 
He refused to read the newspaper or watch television 
because he said he had seen it before. The sensation of 
deja vecu was extremely prominent when he went for 
a walk - AKP complained that it was the same bird in 
the same tree singing the same song, for instance. He 
also read car number plates and stated that the drivers 
must have very regular habits always passing by at the 
exact same time every day. When shopping, AKP would 
say that it was unnecessary to purchase certain items. 


because he had bought the item the day before. There 
was also further evidence of confabulation, including the 
belief that he had been married three times to the same 
woman, with three different ceremonies around Europe. 

On formal testing, AKP had an above average IQ, scor¬ 
ing above the 90th percentile on some of the subtests. He 
scored poorly on some, but not all, tests of frontal function. 
His scores on standardized memory tests, however, were 
impaired, as were his scores on laboratory tests. Consistent 
with his condition, he would mistake the lures for targets. 
As in real life, he not only found new events familiar, but 
actually believed that he experienced them before. 

Structural neuroimaging revealed atrophy of the 
medial temporal lobes but frontal lobes that appeared 
normal. Metabolic measures also showed only medial 
temporal abnormality. This medial temporal abnormality 
may cause 'memory' signals to be emitted continuously 
which tag ongoing perceptions as memories. 


3 are remembered consciously, in such as way that we 
seem to be able to re-experience them 

4 are susceptible to forgetting 

5 are context-dependent, with respect to time, space, 
relationships with others, and other circumstances. 

In contrast, semantic memories (Figure 9.22) 
generally: 

1 have reference to shared knowledge with others 

2 are not organized around a specific time period 

3 give a ' feeling of knowing' rather than a fully con¬ 
scious recollection of the original event 

4 are less susceptible to forgetting than specific 
episodes 

5 are relatively independent of context. 

To investigate the types of consciousness in memory 
tasks, Tulving (1985) introduced the remember/ 
know procedure. This involves asking participants 
to introspect about their conscious experience when 
they recognize studied items. If they believe an item 
was studied before, they must decide whether they 
remember the item (i.e. they can re-experience episodic 
details about the event) or whether they know the item 
(it feels familiar) (see Figure 9.23). Local hippocampus 
seems to affect only 'remember' judgments. Memory 
based on feelings of knowing is spared (Moscovitch 
and Me Andrews, 2002; Yonelinas, 2002). Similarly, in 
functional neuroimaging studies, hippocampal activa¬ 
tion is associated more with remembering than famili¬ 
arity (Eldridge et ah, 2002; Yonelinas et al. , 2005). 

Thus, the hippocampally mediated memory trace is 
suffused with the consciousness that accompanied the 


original experience, or what Tulving called autonoetic 
consciousness. This is contrasted with simply having 
a sense of familiarity about an event, which Tulving 
refers this as noetic consciousness. Noetic consciousness 
is associated with semantic memory. 

4.2 Episodic memories may turn into semantic 
memories over time 

In 1958, Penfield and Milner wrote that: 

The record of the stream of consciousness . . . depends 
upon the integrity of the bilateral hippocampal struc¬ 
tures. . . . Later on, a person deals with what may be 
called generalizations, and he can summon them to his 
purposes. All events, even 'memorable' ones, slip away 
from the reach of voluntary recall unless he has talked 
about them or preserved them by reflective reconsid¬ 
eration. For example, one remembers a song or a poem 
that one has heard repeatedly, forgets each hearing or 
reading, but remembers the generalization, (p. 494) 

More than five decades later many memory scientists 
would agree. There is good evidence that semantic mem¬ 
ories may be formed from repeated, similar episodes. 
Attending a high school is a long series of episodes. 
We may be able to recall dozens of those episodes, but 
much of the time they seem 'smeared' together in mem¬ 
ory in the semantic belief that 'I attended such-and-such 
high school'. 

Figure 9.24 shows how episodic and semantic 
memories may be related in the brain. Specific epi¬ 
sodic memories are shown in the cartoon below: a 
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FIGURE 9.23 Even-related potentials for remembering versus knowing. The act of remembering (recollect¬ 
ing the original experience) results in much higher brain activation than the "feeling of knowing", even for the 
same material. These brain images involve event-related potentials, scalp EEG traces averaged over trials. This 
is consistent with evidence discussed in Chapter 8 indicating that conscious stimuli evoked widespread forward 
activity, outside of sensory cortices, while unconscious stimuli evoke purely local activity Recollecting original 
experiences may involve more consciously retrieved material, and it may also require more mental effort, which 
evokes high activity in prefrontal cortex (see Chapters 10,11, and 12). Source: Duzel et ah, 1997. 


NEOCORTICAL MODULES 



FIGURE 9.24 How semantic and episodic memories may be 
related: semantic memories may be the neocortical residue of many 
episodic memories. Thus one may have many experiences with the 
stick figure above, such as watching him or her cooking a barbecue, 
presenting flowers to a young lady, painting, and playing golf. Over 
time, these episodes may be forgotten, and only the semantic knowl¬ 
edge remains, that this is the kind of person who does all those things. 
These are multiple traces, created each time there is another episodic 
experience with this person. Semantic memory may require only neo¬ 
cortex (particularly temporal and frontal lobes). Episodic information 
may require both neocortex and the hippocampal complex. Source: 
Moscovitch, 2004, modified with permission. 
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man cooking on a barbecue grill, presenting flowers 
to a young lady, painting a picture, and playing 
golf. These are separate autobiographical memories, 
remembered as conscious events. Above, a small 
semantic network combines all these very specific 
and richly detailed episodes into a single figure: a 
semantic network of a man who BBQs, loves, paints 
and plays golf. The semantic network is more abstract 
and general than the episodes about particular events 
in the life of the person. Moscovitch (2004) claims that 
the bottom row of episodes depends upon the MTL, 
and the top figure depends upon neocortical modules. 

We can summarize this concept in three steps: 

1 Initially, memories are episodic and context- 
dependent 

2 Over time, episodic memories are transformed into 
semantic memories 

3 MTL is important for recovering episodic memo¬ 
ries, which are linked to the specific autobiograph¬ 
ical context in which they were acquired. 

In this view, the hippocampal complex or MTL 
supports: 

1 Storage and retrieval of detailed, remote autobio¬ 
graphical and spatial memories 

2 Formation and assimilation of semantic memory in 
neocortex. 

4*3 Episodic and semantic memory are often 
combined 

When you are asked if the face of a particular person 
is familiar to you, say, the movie star Brad Pitt, on 
what basis are you making your judgment? Are you 
relying on your semantic memory, episodic memory, 
or both? Westmacott et al (2004) have shown that both 
systems may contribute, because performance on 
semantic tests is better if the participant also has some 
episodic memory associated with the famous name. 

Semantic dementia is diagnosed in a subset of 
Alzheimer's disease patients who show widespread 
deficits in understanding meaning, often with spared 
perceptual abilities. These patients, whose degeneration 
of anterior and lateral temporal lobes leads to semantic 
loss, can identify common objects only if they can make 
a personal association to them. For example, they can 
identify a vase if it belongs to them, but not any other 
vase (Snowden et al., 1996; Graham et al, 1999). 

Remarkably, however, if MTL is injured very early in 
development, during infancy, semantic memory devel¬ 
ops relatively normally in most cases (Vargha-Khadem 


and Mishkin, 1997). However, episodic memory still 
remains impaired without an MTL. What these findings 
suggest is that early in life, the semantic system has the 
capacity to acquire knowledge on its own, without the 
help of the episodic system (MTL). 

5*0 MTL IN EXPLICIT LEARNING 
AND MEMORY 

MTL is necessary for conscious recollection of long¬ 
term, episodic memories. We do not have a way vol¬ 
untarily to 'switch on' our MTL. Rather, all we need 
to do is to pay attention to some material we want to 
memorize. As pointed out in Chapter 8, that means in 
most cases that we become conscious of the material, 
and an episodic memory is apparently established 
without doing anything more. 

That is, conscious experiences, such as the sight of 
the coffee cup in Figure 9.4, appear to be necessary and 
sufficient to record a conscious experience, using MTL, 
assuming of course that we are dealing with an intact 
brain. The episodic memory trace consists of an ensem¬ 
ble of MTL and neocortical neurons, while the MTL 
acts as a pointer to the neural elements in neocortex for 
the event. Retrieval occurs when a conscious cue trig¬ 
gers the MTL, which in turn activates the entire neo¬ 
cortical ensemble associated with it. When we recover 
episodic memories, we recover conscience experiences 
(Moscovitch, 1995). The recovery of these consciously 
experienced events, in rich detail, always depends on 
the hippocampus no matter how long ago the memory 
was acquired. As Moscovitch (1992) has argued, the 
hippocampal complex acts as a module whose domain 
is consciously apprehended information. Much of the 
evidence seemed consistent with that view. 

Some recent studies, however, have questioned this 
hypothesis. Using fMRI, Henke and her collabora¬ 
tors (Henke et al, 2003; Degonda et al, 2005) showed 
that the hippocampus can be activated by subliminal 
presentation of faces and their associated professions. 
Moreover, these activations are correlated with per¬ 
formance on subsequent explicit tests of memory for 
faces-profession pairs. Likewise, Daselaar et al (2006) 
found that the posterior medial temporal lobe was 
activated more by old, studied items at retrieval, even 
when the person was not aware that the item was old. 
Finally, Schedon et al (2003) showed that the hippo¬ 
campus was activated on the SRT task if the repeated 
sequences were of a higher order of association. 

There also have been similar reports from studies 
with amnesic patients. Ostergaard (1987) was the first 
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to suggest that performance on some priming tests 
was related to the extent of medial temporal damage. 
More recently, Chun and Phelps (1999) showed that 
non-conscious context effects in visual search were not 
found in amnesic patients, suggesting that the MTL 
was needed for retaining contextual information of 
which the person was not aware. Likewise, Ryan and 
Cohen (Ryan et ah, 2000) showed that amnesic people 
did not show the normal pattern of eye movements 
around the location where a change occurred in a 
studied picture, even though neither they nor the nor¬ 
mal people were consciously aware of the change. 

These are a handful of studies in an armful of older 
studies which claimed that the hippocampus is associ¬ 
ated only with explicit memory. If the recent studies 
are replicated and not found wanting, then it would 
change our ideas about the relation of the hippocam¬ 
pus to consciousness and memory. 

5*1 Divided attention interferes with learning 

Learning works best when you pay attention. Trying 
to study in an environment where lots of other inter¬ 
esting things are happening is not likely to work. 
Psychologists have used this 'divided attention' or 
'dual task' technique to understand the contribution 
of attention (or consciousness) to memory. In a typical 
study, participants are asked to process target mate¬ 
rial, such as words or pictures, while at the same time 
their attention is diverted with another task, such as 
tracking a dot on the screen, or deciding whether a 
running count of digits contains three successive odd 
numbers. Under such conditions, even if the partici¬ 
pant is given a task that requires in-depth, meaningful 
analysis of the material, memory under divided atten¬ 
tion is much worse than memory under full attention. 
Successful encoding requires a level of attention and 
presumably consciousness. 

Exactly why that is the case is not well understood. 
One possibility is that deeper processing requires time 
to complete, and divided attention limits the time 


allotted to encoding. Another possibility is that con¬ 
sciousness or awareness is a necessary contributor to 
memory. If one is not fully conscious of the processed 
material, no matter how deeply it was processed, 
memory will suffer accordingly. A third possibility is 
that attention limits elaboration or organization, both 
of which contribute to good memory. 

Fletcher et ah (1995) found, in a PET study, that acti¬ 
vation of the left inferior prefrontal region is reduced 
under divided attention. This finding was replicated 
by Anderson et ah (2000) in younger and older adults, 
with the additional observation that divided attention 
also reduced activity in the left medial temporal lobes, 
regions known to be important for verbal memory. 

Memory and learning have both conscious and 
unconscious aspects. If we think about three phases - 
learning, retention, and retrieval - we can lay out the 
possibilities in a 3 X 4 matrix (Table 9.1). Of the three, 
retention is generally viewed as unconscious, although 
it is shaped by conscious experiences. Learning is 
often thought to require consciousness and, intuitively, 
we certainly try to learn things by paying attention 
and becoming conscious of what we need to learn. 
That is perhaps the basic learning strategy we have as 
human beings. 

However, there is some evidence for learning with¬ 
out consciousness, especially in the case of emotional 
stimuli. There is much stronger evidence for implicit 
learning, in which some inferential process takes con¬ 
scious input and encodes unconscious results of con¬ 
scious input. However, implicit learning tasks always 
ask subjects to pay attention and become conscious of a 
set of stimuli (Section 2.4). It is the rules and regularities 
that generate those stimuli that are learned without con¬ 
sciousness, just as we normally learn the rules of linguis¬ 
tic grammars without knowing those rules explicitly. 
But we must hear spoken words and word sequences 
consciously in order for implicit learning to occur. 

The terms implicit and explicit memory are used 
in the context of remembering, i.e. retrieval of stored 
information. Explicit memory refers to memory with 


TABLE 9.1 Some types of learning and memory - explicit and implicit 


Type 


Learning 


Retention Retrieval 


Episodic memory 
Semantic memory 
Implicit learning 

Subliminal learning (rarely 
robust and long term) 


Conscious or explicit Unconscious 

Conscious or explicit Unconscious 

Conscious stimuli, but unconsciously Unconscious 
learned regularities 

Unconscious of target stimuli Unconscious 


Conscious or explicit of the learning experience 
Unconscious or implicit for the learning experience 
Unconscious retrieval 

Unconscious retrieval 
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conscious awareness, namely, memory of which 
the individual is aware, can declare its existence, 
and comment on its content, either verbally or non¬ 
verbally (Schacter, 1987). For this reason, such memo¬ 
ries also are known as declarative memories (Ryle, 1949; 
Cohen and Squire, 1980). They are the kind of mem¬ 
ory to which we typically refer in everyday conver¬ 
sation when we ask 'Did you remember to call your 
aunt to thank her for the birthday present?' or 'Do 
you remember who won the Academy Award for Best 
Actor or Actress?' 


6.0 PREFRONTAL CORTEX, 
CONSCIOUSNESS, AND WORKING 
MEMORY 

The prefrontal cortex (PFC) plays a critical role in 
working memory. The prefrontal cortex is situated in 
front of the motor cortex in both humans and other 
primates (Figure 9.25). The macaque monkey has been 
the primary experimental animal in many studies of 
working memory. Obviously, humans have other 
abilities, like language, that are not directly paralleled 
in other species. But, in the case of working memory 
studies, the macaque has been a constantly important 
source of evidence. 

Knowledge of a link between the PFC and short¬ 
term memory dates back to the 1930s, when it was 
first discovered that large bilateral lesions of the 
PFC in animals impaired performance on a delayed 
response task. In this task, a sample stimulus is pre¬ 
sented (e.g. a color or location), and its identity must 
be maintained over a short delay period so that it can 
guide a later response (Figure 9.26). Using variants of 
this basic task with more recent neuroscientific tech¬ 
niques, modern research has firmly established the 
role of the PFC in active maintenance of WM informa¬ 
tion (Figure 9.27). 

Much of the animal research has focused on a spe¬ 
cific frontal region called the dorsolateral prefrontal 
cortex (DL-PFC, see Figure 9.25). (In the human cortex 
dorsal is 'upper' and lateral means 'to the side'.) One 
of the key early findings came from the laboratory of 
Joaquin Fuster (Fuster and Alexander, 1971). Fuster 
and his colleagues trained monkeys to perform a 
delayed-response task in which they had to remember 
a color over a brief delay, and then point to the correct 
color when later presented with two alternatives. Since 
no information about the correct color was offered after 
the initial presentation, its identity had to be retained 


in working memory. Using implanted electrodes to 
record neural activity during performance of the task 
(see Chapter 4), it was found that individual neurons in 
the monkey DL-PFC exhibited sustained and persistent 
activity across the delay period. That is, after the color 
had been removed from the visual display, neurons in 
the DL-PFC continued to fire at an increased rate, and 
this activity then subsided once the match/non-match 
response was made (Figure 9.26). 

This pattern of sustained delay-period activity in 
the DL-PFC has been replicated many times since, 
and in a wide variety of tasks. For example, to confirm 
that PFC contributions are truly memory-related, and 
not simply a reflection of subtle preparatory motor 



FIGURE 9.25 The prefrontal cortex in monkeys (top) and 
humans (bottom). The most common division is between upper 
and lower halves of the prefrontal cortex (PFC), called the dorso¬ 
lateral prefrontal cortex (DL-PFC) for the light purple region, and 
the ventrolateral prefrontal cortex (VL-PFC) for the light green area. 
Also notice the orientation cross, pointing to dorsal (upper), ventral 
(lower), rostral (toward the nose in humans), and caudal (toward 
the back of the head in humans). Source : Ranganath, 2006. 




332 


9. LEARNING AND MEMORY 



ini ii i ii in ■ mi 111 i in in ■■ iiifliini 
ii ii ni •aim ■■hi ■ in ■mi i ■ mini 
111 ... iihiiibi mi mil l 11111111 \m nil na 

■HI IB I I llllll II III 11 !■ III! ■I I 1 IIIB I MIBI 1 

mil I i mi mi 11 ii I u ni 

ini i in ii.. ihiiibiii bib ii i ■ 

ii rn mi bmiibiiiihh nan 11 iiiiimbbm 
it a t ii it in i nan ii in in m\m miiii 


11 it 


in 


mi 


ii ai in mi it i i i in n ii I 

HIM HU Util II Bill II Mil III 

iiiiHiii lam a an n m linn n n in 
Hiiaai a n iiiiiii mmn mi ii urn hi ■ i 
i mi hibbi iiiiiii uni i ni i in i 
iia is iiiiiii in a k mu mu am uni a 

““ iiiiiii mi ii ii iaimi mi 

in mi in hi i min n i ii 


ii 


him jib 1 1 in 1 1 n i min i mai Bin ii ii iiiiiii hi iiifiaui n in iiib ii i i m n 

i ii i ammimifli m\§ ■iiiiiii .. mu muni m iiiiiii ii. I muni am ai 

n i ii i 1 1 m\ i laain ni n n in ii 11 ii i mi mn i bib mu inn i 1 1 ib n i mi i n n 

■INI III! I 111 I MIB IB III IBM BIBB III III! IIIIIHI li 111 11 IIIIIIB II I ■■ 1 II I 11 Bl 111 I 

iiiiiii iiiii tii in a sail na iiiiiiin a BiiBiiia ii laini mini . ti i mn ■ i i 

a 11 ii ii 111 p ii ■ 9 i ia ii i mi ii hi ai mi tun 111111181111111 iiiii 111 mi 1111111 
11 1111111111 (inn • 11BM a 1 Bin man n wmt 1 imiiii ■ 10 isnai na mm\ 1 ai 1111 
ill ill ill ill 1 inn in in iiiii iibii 11 1 ■1111 fliai in 111 iiiaiiiiBii 1111 mm ■ ■ in 1 in 11 1 


H 


(b) 



FIGURE 9.26 "Delayed match to sample' in the macaque. In a classic experiment, a macaque mon¬ 
key is trained to delay responding to a stimulus, in this case the location of a red, white, or blue light. 
The monkey shows recognition of the stimulus after delay by matching it in the display, in a task called 
'delayed match to sample'. In effect, the monkey is communicating 'this is what I saw'. DMTS methods 
are widely used in animals, non-verbal babies, and other subjects. Source : Fuster, 1997. 
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(b) Human 


FIGURE 9.27 A delayed-response task to study working memory in monkeys and 
humans. It has been proposed that the PFC serves a specific role in the active storage of 
information in working memory (e.g. Goldman-Rakic, 1998). That is, sustained activity in 
prefrontal neurons reflects this region's role in maintaining specific representations of the 
items that must be kept in mind over the delay. This interpretation is supported by the find¬ 
ing that individual neurons in the PFC are selective for particular target stimuli. For exam¬ 
ple, a given cell may fire strongly over the delay period when the target is in the upper left 
portion of the display, but weakly when the target is elsewhere in the display. This pattern 
suggests direct involvement in the internal representation of target features. In this figure, 
neurons in prefrontal cortex respond during the delay period in a delayed-match-to-sample 
task. Results in macaques and humans are similar. Source: Curtis and D'Esposito, 2003. 


gestures, Patricia Goldman-Rakic and her colleagues 
developed a version of the task in which monkeys 
see a target presented briefly at one of several possi¬ 
ble locations on a display, and then after a delay, must 
shift their gaze to that location in order to receive a 
reward. Importantly, the monkey is required to look 
straight ahead until the end of the delay period, so 
neural activity during the delay cannot be simply a 
byproduct of moving the eye, but must instead reflect 
memory processes. Again, this paradigm produces 
sustained neuronal activity in the DL-PFC and, what's 
more, the amount of delay-period activity predicts 
whether or not items will be remembered; when DL- 
PFC delay-period activity is weak there is a greater 
likelihood of forgetting (Funahashi et al., 1993). 

There is debate whether PFC is subdivided accord¬ 
ing to the content of the information that is stored or 
according to the function that each region carries out. 
According to the content approach, the DL-PFC seems 
to be particularly involved in holding onto informa¬ 
tion about spatial locations, whereas different parts 


of the ventral and lateral PFC have been implicated in 
storing non-spatial types of information (e.g. objects, 
faces, words, etc.). Alternatively, each of these regions 
may have different functions, with DL-PFC implicated 
in manipulation of information and VL-PFC in main¬ 
tenance (ventro-lateral, downward and to the side). 
The term ventral refers to the down direction in cor¬ 
tex; literally, 'ventrum' means 'belly' in Latin. 

Monkey brain lesion studies have further impli¬ 
cated the PFC, and the DL-PFC in particular, in work¬ 
ing memory function. With very precise techniques 
for localizing experimentally-induced lesions, it has 
been shown that damage isolated specifically to the 
DL-PFC is sufficient to impair performance on work¬ 
ing memory tasks (Fuster, 1997). Such findings show a 
causal role for the PFC in working memory. Not only 
are cells in this region active during a delay, but their 
lesioning impairs working memory. This impairment 
gets worse as the length of the delay increases, sug¬ 
gesting that there is more rapid forgetting when the 
PFC is prevented from sustaining them. 
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Studies in humans using neuroimaging have cor¬ 
roborated many of the findings from the animal lit¬ 
erature. Hundreds of imaging studies have shown 
PFC activity when participants are trying to maintain 
task-relevant information. Consistent with the animal 
work, fMRI studies in humans show that PFC activity 
persists during the delay period of a working memory 
task (see Chapters 4,10, and 12; and the Appendix). 

Human neuroimaging studies have also varied 
working memory load - the number of items that must 
be held in immediate memory (Cohen et ah, 1997; 
Rypma et ah, 2002). In one study, memory load was 
varied between one and eight items, and subjects 
had to hold these items for a short delay. PFC activa¬ 
tion was found to be positively correlated with the 
number of items in memory. Such 'load dependence' 
in the PFC supports the notion that this part of the 
brain is involved in working memory storage (See 
Chapter 12). 

While PFC contributions to working memory have 
been clearly demonstrated, its specific contribution 
to working memory storage has been recently ques¬ 
tioned. Several other cortical and subcortical areas 
exhibit similarly persistent stimulus-specific activity 
over short delays. It appears that PFC may be part of 
a more distributed brain network supporting working 
memory. Other data suggest that the PFC may not be 
involved in storage per se, but in providing top-down, 
or executive, support to other regions where informa¬ 
tion is actually stored. 


6*1 Working with memory: the frontal lobe 
works purposefully with memory 

According to this model, both encoding and retrieval 
of consciously apprehended information via the hip¬ 
pocampus and related structures is obligatory and 
automatic, yet we know from experience and from 
experimental investigation that we have a measure of 
control over what we encode and what we retrieve from 
memory. Moreover, if encoding is automatic and obliga¬ 
tory, the information cannot be organized, yet memory 
appears to have some temporal and thematic organi¬ 
zation. How can we reconcile this model of memory 
with other facts we know about how memory works? 
One solution is that other structures, particularly those 
in the frontal lobes, control the information delivered to 
the medial temporal system at encoding, initiate and 
guide retrieval, and monitor and help interpret and 
organize the information that is retrieved. By operating 
on the medial temporal and diencephalic system, the 


frontal lobes act as working-with-memory structures that 
control the more automatic medial temporal system 
and confer a measure of intelligence and direction to it. 
Such a complementary system is needed if memory is to 
serve functions other than mere retention and retrieval 
of past experiences (Moscovitch, 1992). 

6*2 Prefrontal cortex in explicit (conscious) 
and implicit (unconscious) learning and 
memory 

Figure 9.5 pointed out that working memory may 
help us to learn both explicit (conscious) and implicit 
(unconscious) information. One of the functions often 
attributed to consciousness is the integration of infor¬ 
mation across domains. In a very illuminating study, 
McIntosh et al. (1998) had subjects perform in a trace 
conditioning task which requires the person to make 
an association between a color and a tone separated 
by a blank delay of about a second. Previous work 
had shown that such conditioning is dependent on 
the hippocampus. Moreover, Clark and Squire (1999) 
showed conscious awareness of the association was a 
prerequisite for this kind of learning. 

Using PET, McIntosh showed that learning, and the 
conscious awareness that accompanied it, was associ¬ 
ated both with frontal activation and with coherence 
of activation across many areas of cortex. McIntosh 
et al. speculated that consciousness is associated with 
activation in prefrontal cortex which, in turn, leads to 
a correlated pattern of activity across disparate regions 
of cortex. It remains to be seen, however, whether 
frontal activation preceded or followed conscious 
awareness of the association. To do so, it is necessary 
to have a clear understanding of the sequence of acti¬ 
vation across regions of cortex, using techniques that 
have a higher temporal resolution than fMRI, such as 
ERP or MEG. As yet few, if any, studies have used ERP 
and MEG to address issues regarding the time course 
of consciousness as it relates to memory. 

If the prefrontal cortex plays a pivotal role in con¬ 
sciousness, as many people have speculated, deficits on 
all memory tests dependent on consciousness should be 
observed in patients with frontal lesions. However, so far, 
the evidence indicates that the effects of frontal lesions 
are much more selective and not nearly as debilitating as 
lesions to MTL and related nuclei of the thalamus. 

The prefrontal cortex contributes to performance 
on implicit learning and memory if it requires search, 
sequencing, organization, and deliberate monitor¬ 
ing. Implicit learning of language is a good example 
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BOX 9.4 Implicit learning of language. While we are given the words and sequences of language, the rules and 
regularities of grammar and perhaps meaning are often inferred unconsciously, i.e. using implicit learning 


Implicit learning and language acquisition 

The past few years have witnessed the emergence of 
increasing connections between implicit learning and 
psycholinguistics. This is perhaps not so surprising, in 
that language acquisition, like implicit learning, involves 
incidental learning conditions. Further, cogent use of 
language likewise does not require explicit knowledge 
of grammar. Recently, several authors have begun to 
explore this connection empirically. For instance, Saffran 
et al., 1997, showed how incidental exposure to artificial 
language-like auditory material (e.g. bupadapatubitutibu 
. . .) was sufficient to enable both children and adult sub¬ 
jects to segment the continuous sequence of sounds they 
had heard into the artificial words (e.g. bupada, patubi, etc.) 
that it contained, as evidenced by their above-chance per¬ 
formance in a subsequent recognition test. Based on these 
data, Saffran et al. suggested that the word segmentation 
abilities demonstrated by these subjects were due to the 
transitional probabilities of successive syllables are higher 
within words than between words. Saffran and colleagues 
interpreted their findings as representing a form of implicit 
learning. The connection is obvious when one recognizes 
that language acquisition, like implicit learning (Berry 
& Dienes, 1993; Cleeremans, 1993) is likely to involve, 
at least in part, incidental learning of complex informa¬ 
tion organized at differing levels. Part of the conver¬ 
gence between language acquisition and implicit learning 


suggested by Saffran and colleagues can be attributed to 
the impact of computational modeling on the field of mem¬ 
ory research. For instance, connectionist models such as 
the Simple Recurrent Network have been extensively used 
with significant success in both the language acquisition 
and implicit learning domains (Christiansen et al ., 1998; 
Redington & Chater, 1997). In effect, the problems faced in 
both domains are quite similar: how to best extract struc¬ 
ture from a complex stimulus environment characterized 
by 'deep' systematic regularities when learning is inciden¬ 
tal rather than intentional. The answer, in both domains, 
appears to be embodied by distributional approaches. 
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(Box 9.4). Even though we rarely try to make the rules 
of grammar conscious and explicit, we nevertheless 
need to direct our attention to the order of words in 
a sentence, for example, to learn a language implic¬ 
itly. It is likely that unconscious inferences help us to 
discover rules or regularities, provided that we pay 
'conscious attention' to a series of words, for example, 
from which we can discover the implicit regularities. 

63 Different types of working memory 

In proposing the concept of working memory, Baddeley 
and colleagues (2004) reasoned that subjects should 
have difficulty keeping items in memory if asked to 
perform other tasks simultaneously that disrupt spe¬ 
cific components of the working memory system. 
Consistent with this view, Baddeley and others have 
shown that asking subjects to repeat aloud a simple 
utterance (like the word 'the') during retention can dra¬ 
matically reduce short-term memory for verbal infor¬ 
mation, presumably because the repetitive speech task 
disrupts the rehearsal mechanism of the phonological 
loop. Importantly, the same repetitive speech task has 


a much smaller effect on working memory for visu- 
ospatial information because holding such information 
does not make as much use of the phonological loop. 
Meanwhile, other tasks (e.g. visually tracking a moving 
object) have been found to disrupt visuospatial, but not 
verbal, maintenance. 

We have already encountered examples of patients, 
like Clive Wearing and HM, who seem to have spared 
short-term memory but impaired long-term memory. 
Elizabeth Warrington and Tim Shallice (Shallice and 
Warrington, 1970) reported one of the earliest recog¬ 
nized cases of a patient, KF, with the opposite pat¬ 
tern of impairment - a severely impaired short-term 
memory but apparently intact long-term memory. For 
example, when asked to recall short lists of spoken 
digits, the digit-span task , KF could recall only one or 
two items reliably (as compared to a typical digit-span 
of around seven items). Still, KF had comparatively 
normal speech-production abilities, and could learn 
and transfer new information into long-term mem¬ 
ory. The finding that a patient with severely impaired 
short-term memory could still transfer information 
into long-term memory presented a challenge to the 
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standard hypothesis posited that a unitary short-term 
memory serves as the gateway into long-term memory. 
Baddeley's working memory model suggested that if 
verbal rehearsal is impaired, the visuospatial sketch¬ 
pad might be used to compensate (see Figure 9.5). 

Indeed, the short-term memory impairment in 
patient KF, and a number of similar patients reported 
since, seems to be tied to particular types of informa¬ 
tion. For example, while these patients struggle to 
remember verbal items when presented auditorily, 
their performance is considerably improved when 
the items are presented visually. What might account 
for this pattern of findings? Baddeley's answer is that 
visually presented items can be coded directly into the 
visuospatial sketchpad, thus avoiding the damaged 
verbal rehearsal loop. 

Neuroimaging has helped to clarify different kinds 
of memory. These include the distinction between 
verbal and visuospatial maintenance subsystems 
(e.g. Smith et ah, 1996), the dissociability of storage 
and rehearsal in verbal maintenance (Paulesu et ah, 
1993; Awh et al., 1996) and the assumption of a central 
executive processor that mediates the behavior of the 
subsidiary maintenance subsystems (e.g. Curtis and 
D'Esposito, 2003). In general, neuroimaging studies 
have tended to support the basic model (Smith and 
Jonides, 1998; Hartley and Speer, 2000; Henson, 2001). 

Figure 9.28 illustrates the brain network impli¬ 
cated in neuroimaging studies of WM (Curtis and 
D'Esposito, 2003). The central executive corresponds 
to DL-PFC (labelled as 'D' on the figure), the verbal 



FIGURE 9.28 Brain areas believed to be involved in verbal 
and visual working memory. A simplified brain model of working 
memory (Figures 9.4 and 9.5). Abbreviations are D = dorsolateral 
prefrontal; B = Broca's area, also called the left inferior frontal 
gyrus (L-IFG); P = phonological loop for verbal rehearsal, also 
called the supramarginal gyrus; and F = frontal eye fields, believed 
to be involved in the visuospatial sketchpad of Baddeley's working 
memory model. Source : Curtis and D'Esposito, 2003. 


maintenance subsystem in left lateralized regions of 
the temporo-parietal junction (labelled as 'P' on the 
figure) and VL-PFC, and the visuospatial mainte¬ 
nance subsystem in the superior parietal cortex, pos¬ 
terior PFC, and frontal eye fields (FEF, labelled as 'F' 
on the figure). However, we should be cautious in our 
acceptance of these findings as direct support for the 
framework, since few researchers have considered 
the applicability of these findings to alternative theo¬ 
ries of working memory (Chein et ah, 2003; Ravizza 
et ah, 2005). In addition, the neuroimaging literature 
has at times challenged aspects of the standard work¬ 
ing memory model. For example, neuroimaging evi¬ 
dence suggests that different types of visuospatial 
information may depend on different storage subsys¬ 
tems. For instance, there seem to be different neural 
substrates for the maintenance of object information 
as compared to the maintenance of spatial locations, 
a distinction that is not addressed by the traditional 
working memory model. (Recall that this 'what' ver¬ 
sus 'where' dichotomy is important also in the visual 
processing pathways; see Chapter 6.) 

6A Prefrontal cortex - storage or process 
control? 

In the beginning of this section on working memory, 
we reviewed several sources of evidence suggesting 
that the PFC is an important site for working memory 
function. According to one interpretation, this brain 
region participates directly in the storage of informa¬ 
tion. However, consideration of findings in the context 
of Baddeley's multiple-component model suggests an 
alternative account - namely, that the PFC is more 
closely associated with control, or executive, aspects 
of working memory. 

Specific evidence against a 'storage' interpretation 
of PFC function also comes from studies of humans. 
We discussed earlier a group of patients with left 
temporo-parietal damage who appear to have a stor¬ 
age deficit in working memory, and can't perform 
even simple maintenance tasks with auditory-verbal 
information. The findings from these patients can 
be contrasted with those from patients with dam¬ 
age to the PFC. In a review of published reports from 
patients with damage to large regions of the lateral 
PFC (D'Esposito and Postle, 1999), it was found that 
PFC patients showed little or no impairment on tasks 
that called for the passive maintenance of information 
over a delay (e.g. verbal and non-verbal memory span 
tasks). However, these patients were found to be sub¬ 
stantially impaired at tasks that required information 
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in working memory to be mentally manipulated or 
acted upon. This pattern of findings suggests that the 
PFC serves to support the mental 'work' performed 
on stored information, rather than as a site for storage 
itself. Few of the patients in this review had bilateral 
lesions, leaving open the possibility that storage and 
rehearsal are achieved via bilateral PFC representa¬ 
tions and may thus allow functional compensation 
from the undamaged hemisphere (D'Esposito and 
Chen, 2006). 

One possibility is that different parts of the PFC do 
different things. This proposal has generally focused 
on differences between dorsal (DL-PFC) and ventral 
(VL-PFC) areas. Advocates of this view have argued 
that the PFC is not organized by domain (e.g. spatial 
versus non-spatial), but by process, with ventral areas 
of the PFC supporting the passive storage and main¬ 
tenance of items, while more dorsal areas are called 
upon when the task demands selection, monitoring, 
manipulation, or other 'mental work' to be performed 
on these items. This is the so-called 'maintenance' 
versus 'manipulation' processing distinction. While 
this view seems capable of explaining a wide range of 
findings, several studies have cast doubt on even the 
assumption that the VL-PFC contributes to storage in 
WM (e.g. Rushworth et ah, 1997). 

More recently, it has been argued that all of the PFC 
has an executive function in working memory, but 
that different subdivisions of the PFC perform this 
function at different levels of analysis (Ranganath, 
2006). This emerging view asserts that the primary 
function of the PFC is to modulate the activity of other 
cortical areas where the items in memory are stored. 
Specifically, PFC representations enhance relevant 
information (or inhibit irrelevant information) repre¬ 
sented in other parts of the cortex. When the informa¬ 
tion is specific to individual items in memory, more 
ventral PFC regions are engaged. When the informa¬ 
tion regards the integration of (or relations between) 
multiple items in memory, more dorsal PFC regions 
are engaged. Anterior regions of PFC, at the frontal 
pole, are implicated in coordinating and monitoring 
activity among different PFC regions to implement 
higher-order functions, such as planning. Accordingly, 
the primary role of the PFC is not in working mem¬ 
ory, but in working with memory (Moscovitch, 1992; 
Moscovitch and Winocur, 1992). 


6*5 Combining prefrontal and MTL regions 
for working memory 

Working memory is usually believed to operate over a 
few dozen seconds or minutes. However, even within 


a few minutes' time, we can find differences between 
different kinds of memory. In particular, you may 
have seen the word 'combining' only a minute or two 
ago, and yet you may not be able consciously to recall 
it. We can therefore make a prediction, based on our 
previous understanding of the role of MTL in explicit 
memory: if you can recall seeing the word 'combining' 
a brief time ago, your MTL should show activity in an 
fMRI experiment in which you are asked to recall the 
first word of this section. On the other hand, if you do 
not have an explicit memory of it - even over a short 
time period - your MTL should show less activity. But 
perhaps you will still retain a semantic memory of the 
title of this section, based on your neocortical encod¬ 
ing of that word. 

We can therefore even describe working memory in 
terms of MTL, prefrontal cortex, sensory cortex, and 
the like. Figure 9.29 shows a model of how these brain 


* j 



(a) 




(b) 


FIGURE 9.29 Combined brain regions work together for 
visual working memory. One view of visual working memory sug¬ 
gests that the hippocampus may encode WM items that are novel, 
the wider MTL may combine them with other modalities, and IT is 
involved in high-level visual object representation. The DL-PFCs 
and anterior PFC (purple) is involved with the short-term mainte¬ 
nance of relations, while the VL-PFC and posterior PFC is involved 
with maintenance of relevant items. As pointed out in the text, this 
is only one current hypothesis about the functions of these regions. 
However, it is widely believed that some network of functions like 
this may be needed to give a complete account of working memory. 
Source : Ranganath and D'Esposito, 2005. 
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areas may interact, based on a large number of studies 
of this kind (Ranganath and D'Esposito, 2005). Many 
of the specifics of this model are still debated, and we 
can expect new studies to cast some light on them. 
There seems to be an emerging consensus, however, 
that a complete explanation of working memory func¬ 
tions will require a multiregional model of this kind. 


7*0 RETRIEVAL AND METACOGNITION 

Clive Wearing knows that something is terribly wrong, 
but he has no idea what it is. Tve just woken up for 
the first time. I'm conscious for the first time' is his 
only way to express it. For more than 20 years he has 
expressed the most intense frustration with his condi¬ 
tion. Wearing must therefore have some metacognitive 
conception of his own cognitive functioning, unlike 
patient HM, for example, who is spared the emotional 
pain of sensing what he is missing. Metacognition is 
defined as the ability to know our own cognitive func¬ 
tions, and to be able to use that knowledge. Many neu¬ 
rological patients who are severely impaired have no 
metacognitive insight that anything is wrong (Milner 
and Rugg, 1992). 

For retrieval to be effective, information at retrieval 
must overlap with the information that was learned or 
encoded. In addition, the person must have the goal 
of retrieving memories and paying attention to cues, 
as well as mentally searching for the desired memory. 
In all these processes, monitoring and verification are 
necessary, as is coordination of the various activities. 
Different regions of prefrontal cortex are implicated in 
many of these processes. The MTL is implicated most 
strongly with retrieval success for episodic memories. 
Automatic retrieval of a memory often occurs once 
the cue is found (Figure 9.30). Strategic or purpose¬ 
ful retrieval, however, is attention demanding and is 
impaired by any effortful competing task. 

Metacognition is an important aspect of normal 
memory retrieval. A memory trace may be retrieved 
spontaneously, or more often by cues or reminders. A 
cue could be as simple as 'Recall the words you just 
studied' or as complex as 'Describe in detail what you 
did today'. The kind of self-monitoring we tend to do 
when we try to remember a missing word is a kind of 
metacognition that involves prefrontal cortex. 

Schacter and colleagues (Schacter et al., 1984) 
showed that memory for source in amnesic patients 
was related to the extent of their accompanying fron¬ 
tal deficits, rather than the severity of their amnesia. 


NEOCORTICAL MODULES 



FIGURE 9.30 Retrieving semantic memories by using epi¬ 
sodic cues and vice versa. We can often retrieve a semantic memory, 
like a fact about the world, by being cued with an episodic memory; 
the association can also go the other way. Thus episodic and seman¬ 
tic memory continue to be potentially connected, even if they exist 
relatively independently of each other in the hippocampal complex 
(MTL) and neocortex. Notice that most episodic memories for this 
semantic cluster of associations (on top) have faded. Source : Morris 
Moscovitch, personal communication. 

Extending these findings to older adults, Glisky 
(Glisky et al., 1995) showed that those with poor 
frontal function were impaired on tests of source 
memory but performed normally on tests of item 
memory, whereas the reverse was true of those with 
poor medial temporal function. Similarly, damage to 
prefrontal cortex leads to deficits in memory for fre¬ 
quency of occurrence (Smith and Milner, 1984) and 
temporal order (Milner, 1971; Shimamura, 1984) even 
when memory for the item itself is preserved. 

The functional neuroimaging literature is consis¬ 
tent with the lesion literature. In comparison to tests of 
item memory, tests of source memory activate the DL- 
PFC and the frontal pole more strongly (Fletcher and 
Henson, 2001; Fletcher et al., 2005), as do tests of tem¬ 
poral order (Nyberg et al., 1996; Cabeza et al., 1997). 

The role of metacognition, using prefrontal cortex, 
is especially dramatic when it fails. This is the case 
when people cannot recognize their own mistakes, as 
we see next. 


7.1 False retrieval 

There are deficits of retrieval that are quite remarkable. 
The case of confabulation is particularly important, the 
inability of some brain-damaged patients to tell that 
the 'memories' they are retrieving are entirely imagi¬ 
nary. One such case is illustrated in Box 9.5. 
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BOX 9.5 HW, a confabulating patient 


HW is a 61-year-old man who was married to the same 
woman, Martha, for over 30 years and had four grown 
children, ranging in age from 22 to 32. He worked in a 
management position at a factory He had an aneurysm 
of the anterior communicating artery, which feeds the 
medial frontal lobes and basal forebrain. Surgical clip¬ 
ping of the aneurysm was followed by widespread bilat¬ 
eral ischemia (loss of oxygen) and infarction. CAT scans 
confirmed widespread frontal damage, with sparing of 
the temporal lobes medially and laterally 

His brain damage led to memory problems exacer¬ 
bated by confabulations and an unawareness of his mem¬ 
ory deficits. His intelligence was normal, as measured by 
standard tests. HW's confabulations were spontaneous 
and, because he believed they were true memories, he 
attempted to act on them, making it difficult to let him 
live on his own. For example, there were times in the 
hospital that he believed he was at work, even though 
the hospital had beds, nurses, etc., and he wore hospital 
clothes. Many days at 5 pm he would prepare to go home 
and became frustrated and belligerent when he was told 
he had to stay. 


Here is a part of an interview with HW: 

Q: How long have you been married? 

A: About 4 months. 

Q: What's your wife's name? 

A: Martha. 

Q: How many children do you have? 

A: Four. (He laughs). Not bad for four months. 

Q: How old are your children? 

A: The oldest is 32 ... the youngest is 22. (He laughs again.) 
Q: How did you get these children in four months? 

A: They're adopted. 

Q: Who adopted them? 

A: Martha and I.. .. 

Q: Does it all sound a little strange to you? 

A: (He laughs) I think it is a little strange. 

Q: Your record says that you've been married for over 30 
years. Does that sound more reasonable to you? 

A: No. 

Q: Do you really believe that you have been married for 
four months? 

A: Yes. 


7.2 Hemispheric lateralization in retrieval 

In light of the left hemisphere preference for language 
production (Chapter 11), it is perhaps unsurprising that 
memory functions may also be lateralized according to 
the nature of the material being processed. In working 
memory, for example, maintenance of verbal materials 
activates left regions in the parietal cortex and VL-PFC, 
whereas maintenance of non-verbal materials tends to 
be right-sided. Regions in the VL-PFC are also impli¬ 
cated in encoding information into long-term memory, 
and seem to interact with areas of the MTL in serving 
this function. Accordingly, both the VL-PFC and MTL 
have been found to exhibit hemispheric asymmetries 
in long-term memory encoding (Kelley et al., 1998). 

Lateralization may depend upon the materials used 
(verbal, for example), and also upon the task and cog¬ 
nitive process. Endel Tulving and colleagues (Habib et 
al., 2003) proposed that, in general, learning is associ¬ 
ated with greater involvement of the left PFC, while 
retrieval shows greater involvement of the right PFC 
(Figure 9.31). This model has received substantiation 
from a large number of neuroimaging studies. 

Some researchers have questioned, however, whether 
this encoding/retrieval asymmetry may arise simply 
due to a greater dependence on verbal processes during 
learning than retrieval. The hemispheric bias seems to 
hold for episodic memories, while for semantic memory, 
both learning and retrieval seem more dependent on left 
hemisphere mechanisms. 


Encoding 


Retrieval 



Right 


Left 


FIGURE 9.31 Left hemisphere for learning, right hemisphere 
for retrieval? Tulving and colleagues (Habib et al., 2003) found that 
the left hemisphere shows greater activity in episodic learning 
(encoding), while the right side showed more activity in episodic 
retrieval. Source : Habib et al, 2003. 


A recent study by Rossi and colleagues (Rossi et al, 
2006) using transcortical magnetic stimulation (TMS, 
see Chapter 4), a technique that allows temporary 
disruption to cortex in healthy subjects, supports the 
Tulving proposal. Their study showed that temporary 
disruption in the left PFC reduced learning efficiency, 
as measured by recognition accuracy. In contrast, TMS 
over the right PFC reduced retrieval efficiency. Thus, 
there seems to be a relationship between hemispheric 
lateralization and the learning-retrieval distinction. 
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FIGURE 9.32 Theta oscillations may coordinate MTL and the prefrontal lobe during retrieval. Regular 
brain rhythms may serve to coordinate separate brain regions. Hippocampal theta is believed to reflect memory 
retrieval processes, and to coordinate prefrontal cortex with the MTL. Source : Jensen, 2005. 


7.3 Theta rhythms may coordinate memory 

retrieval 

A new literature shows that theta rhythms (5-8 Hz) 
appear in the frontal lobes during memory retrieval. 
Depth electrodes placed in the hippocampi show that 
the MTL and prefrontal regions may be coordinated 
during retrieval (Siapas et al., 2005). Figure 9.32 sum¬ 
marizes these findings, which begin to show how the 
neural networks of prefrontal and hippocampal struc¬ 
tures may cooperate to draw out and coordinate mem¬ 
ory traces during retrieval. 

8.0 OTHER KINDS OF LEARNING 

We have only discussed some kinds of memory. We 
can only briefly mention some others, which make use 
of other brain structures (Figure 9.33). For example, 
the amygdala mediates fear conditioning (see Chapter 
14). The cerebellum and basal ganglia are needed for 
habits and skills, as well as some kinds of condition¬ 
ing. The thalamus is one of the great information hubs 
of the brain, constantly trading signals with cortex. It is 
therefore believed to be involved with cortical learning 
mechanisms. 

Perceptual and motor learning involve the dynamic 
organization and reorganization of cortical maps 


(Chapters 2 and 5). This is called neural plasticity. For 
example, losing a finger will change the motor map 
representing the finger in the macaque cortex. In 
humans, brain surgery can alter sensory body maps 
very quickly, even during the operation. Life devel¬ 
opment itself can be viewed as a process of learning, 
adaptation, and memory formation (see Chapter 15). 
Finally, we now know that new neurons can be born 
throughout the lifetime, starting from stem cells (neu¬ 
ral progenitor cells). The ongoing replacement of some 
neurons also involves a dynamic learning and adap¬ 
tation process. There are many ways for the brain to 
learn. Though we have focused on more standard con¬ 
cepts of learning and memory, it is important to keep 
that in mind. 


9.0 SUMMARY 

The medial temporal lobes (MTL) are crucial to epi¬ 
sodic memory in which we retain information about 
the conscious source of the memory. Amnesia patients 
with bilateral damage to the MTL are correspond¬ 
ingly unable to remember specific past episodes or 
to learn new ones. However, implicit learning and 
memory may be spared in these patients. Patients suf¬ 
fering from semantic deficits typically have damage 
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FIGURE 9.33 Different kinds of memory use different brain regions. An overview of multiple learning 
systems in the brain. The MTL system discussed in this chapter is involved in learning and recalling declara¬ 
tive, explicit memories; in memory consolidation; and the like. The PFC appears to be active in working 
memory tasks, metacognitive memory judgments, semantic memory, and conceptual priming. Perceptual 
priming involves sensory cortex, while motor learning and some kinds of classical conditioning seem to 
require the cerebellum. Source : Morris Moscovitch, with permission. 


in the temporal lobe (especially the anterior and lat¬ 
eral parts) and prefrontal cortex. Such patients with 
semantic dementia, for example, may retain their epi¬ 
sodic memory but are impaired on semantic tasks. 

While explicit memory is assessed by accurate source 
memory reports, implicit methods like priming and 
sensorimotor performance may be needed to assess 
implicit memory types. Much of our learning is implicit, 
such as the learning of language. However, it is impor¬ 
tant to keep in mind that implicit learning requires 
conscious and attentive orienting to the stimuli to be 
learned. What is unconscious about implicit learning is 
not the original stimuli, but the inferential regularities 
that allow us to organize those stimuli. Sensorimotor 
skills are guided by the frontal cortex in collaboration 
with the basal ganglia and cerebellum. After overprac¬ 
ticing predictable tasks, such learned skills become less 
conscious and seem to rely only on subcortical struc¬ 
tures like the basal ganglia. 


Working memory can be decomposed into vis¬ 
ual (the visuospatial sketchpad) and verbal (verbal 
rehearsal or the phonological loop). Further divisions 
are often possible, such as a separation between spa¬ 
tial and visual working memories. 

A complete conception of human memory requires 
multiple brain regions: the MTL for explicit episodic 
memories; the prefrontal cortex for metacognition, main¬ 
tenance, and use of memory; and sensory regions for 
perceptual and sensory memories. The cerebellum and 
basal ganglia are required for sensorimotor skill learning, 
in interaction with the frontal lobes. Further, sensory and 
motor halves of the cortex are in constant dialogue with 
each other, as when we hear ourselves speak. Finally, 
the amygdala is deeply involved in emotional learning, 
along with associated regions in the limbic system. 

A wide range of memory deficits can be used to sep¬ 
arate different memory components from each other. 
You should know some examples of such dissociations. 
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10-0 DRAWINGS AND STUDY QUESTIONS 


1 Fill in the missing labels in the functional diagram given in Figure 9.34. Define each of the terms. 


Functional framework for learning and memory 


Attention 

Vision 

Hearing 


Touch 


Cl 




Response 

output 




FIGURE 9.34 A functional diagram for learning and memory. 


2 Label the regions and associated types of memory. 



FIGURE 9.35 Relevant parts of the cortex. 


3 Label the brain regions that are relevant to 
learning and memory in Figure 9.36. 



FIGURE 9.36 Location of some memory regions. 
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4 Label and describe: 


5 Label the abbreviations in Figure 9.38. What sys¬ 
tem does this describe? 




FIGURE 9.38 Brain areas theorized to be involved in 
specific aspects of memory processing. 


FIGURE 9.37 Location of some memory regions. 











Recruitment of executive attention is normally associated to a subjective feeling of mental effort . 
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High level problem solving. Micronesian navigation is a highly complex and successful type of expertise. If it were not, island peoples of the 
Pacific Ocean could not navigate hundreds of miles in tiny craft to travel from island to island, surviving "the stern test of landfall', as they 
have for thousands of years. The star compass on the upper right is an invention of oceanic navigators, who keep track of their location by not¬ 
ing the angle of rising and setting stars at night with respect to the North Star. High level knowledge of navigation was developed, memorized, 
and passed on by renowned experts. The outrigger sailing craft is a very efficient way to make the long journeys, but hardly without danger. 
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Human problem solving comes in two varieties, 
explicit and implicit. These two modes differ remark¬ 
ably. Explicit thinking has clear, conscious goals and 
subgoals, and clearly defined steps for getting from 
a starting point to a solution. We can think of mental 
arithmetic as an everyday example. Explicit think¬ 
ing involves greater executive control, higher mental 
workload, more frequent conscious access, and wider 
recruitment of cortical regions in pursuit of explicit 
goals. 

In contrast, implicit problem solving may be more 
common, since we learn and practice many kinds of 
skills from early on in life. These problem solving 
skills become more proficient, implicit (unconscious), 
and automatic with practice. Understanding this sen¬ 
tence is an example, or completing a sentence that 
is predictable, so that we can tell what its last words 
are likely to be. Implicit problem solving takes less 


executive control than the explicit kind, less conscious 
access, less cognitive load, and less cortical involve¬ 
ment. It is much more dependent, on the other hand, 
on long-term memory and highly practiced routines. 
Very often, the unstated goal of learning is to turn 
explicit problem solving into the implicit kind. 

In the realm of thinking, the historic debate between 
local and distributed brain functioning continues. 
With neuroimaging methods we can see far more local 
regions and processes than ever before. We might 
imagine therefore that localist theories might finally 
be winning the day, but that has not (yet) happened. 
While ever smaller and more specific brain regions 
are being examined, the evidence for widely distrib¬ 
uted processes is still compelling (see Chapter 9). 
A number of theorists are therefore developing mod¬ 
els that embody both localized and widely distributed 
neural networks. 
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FRONTIERS OF COGNITIVE NEUROSCIENCE 


Focal attention and 

Every problem we try to solve in everyday life creates 
demands on our focal attention - what we are conscious 
of - as well as our immediate memory capacity It also 
requires us to control to what we decide to pay atten¬ 
tion. Nelson Cowan has spent a productive career study¬ 
ing these 'limited capacity features' of the human brain, 
along with their constant interactions with the large 
storehouse of long-term memory (Figures 10.1 and 10.2). 
Cowan writes that 'what appears to be a general capacity 
limit in working memory. . . is closely related to the con¬ 
tents of the conscious mind' (2008). 

Focal attention is even more limited than psychologists 
have traditionally thought, according to Cowan's research. 
When careful experiments are done to exclude mental 
rehearsal and item grouping, the capacity of human focal 
attention comes down to four items or less. The traditional 
view was that immediate memory could hold 'seven plus 
or minus two' separate items, but that holds true only if 
subjects can rehearse the items to themselves. If subjects are 
asked to keep repeating 'the ... the ... the' to prevent them 
from rehearsing memory items, the capacity of immediate 
memory drops to four. The limit of four items has been asso¬ 
ciated with the posterior parietal cortex, but there is ongoing 
debate about the brain basis of limited capacity. 

The 'magic number four' is quite stable across differ¬ 
ent kinds of items - digits, short words, simple sounds, 
visual objects, and so on. Cowan suggests that it applies 
to abstractions as well as perceptual events, images, and 
inner speech. To add two digits in memory we have to 
hold both of them for several seconds while calculat¬ 
ing the sum. To understand a long sentence, we need to 
hold the first half in immediate memory in order to make 
sense of the second half. The sentence before this one is a 
good example. But there is evidence that each meaningful 
chunk of a long sentence is encoded into semantic units: 
Like the phrase, 'To understand a long sentence'. . . 



FIGURE 10.1 Nelson Cowan, PhD, Department of Psycho¬ 
logical Sciences, University of Missouri, Columbia, MO, USA. 


immediate memory 

Humans are very skilled in semantic encoding, because 
we have a lot of practice in the job of understanding what 
people are trying to communicate to us. We tend to con¬ 
vert words into meanings as soon as we can do so. 

The trick is therefore to make the best use of the lim¬ 
ited capacity features we are born with. One approach is 
to rehearse a set of numbers to ourselves in inner speech. 
With rehearsal our working memory limit goes up to the 
traditional 'seven plus or minus two items'. But then we 
are using both our focal attention plus our immediate 
memory capacity for inner speech and mental rehearsal. 

We can also lighten immediate memory load by scrib¬ 
bling a note or highlighting a sentence in a book. But writ¬ 
ing is a recent human invention, not to mention audio and 
video recording. Before the invention of writing, humans 
were still able to solve complex problems, as we know 
from cultures that do not use writing. But they had to rely 
on their long-term memories (LTM), on mnemonics, and 
on highly elaborated knowledge of their world. A lot of 
cultures use speech patterns, like poetic rhymes, or allit¬ 
eration, melodic chants, or poetic meter. That is how young 
children often learn their 'ABC's', through a song. It's not a 
bad way to learn brain anatomy either. 


Embedded-processes model 



FIGURE 10.2 The cognitive psychologist Nelson Cowan 
suggests that immediate memory can be thought of as the acti¬ 
vated parts of long-term stores. For example, repeating words to 
yourself activates existing vocabulary items in your long-term 
knowledge of language. It is interesting that novel associations 
between known elements need to be brought together in the focus of 
attention in order to be encoded in long-term memory. It is impor¬ 
tant to see immediate memory and the focus of attention, inner 
rehearsal, and other aspects of "working memory" as ever-present 
aspects of normal cognition. Thus WM elements may be tempo¬ 
rarily borrowed from LTM, and inner speech may simply utilize 
the "outer speech" capacities of the human brain. Source : Nelson 
Cowan, personal communication. 
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Humans can solve problems with focal attention and 
immediate memory, as long as we combine them with 
our very large libraries of long-term memories. Every 
word you read may take up some limited focal attention, 
for just a moment, but the fact that you are thoroughly 
familiar with about 100,000 words and their meanings 
allows you to process rich 'word chunks', and not just 
meaningless blips and gurgles. 

The part of long-term memory that is engaged with our 
present circumstances Cowan calls active memory. In reading 
this paragraph, your active memory is what you have built 
up in earlier chapters, along with all the active knowledge 
you are bringing to bear from sources. As authors, our effort 
is always to try to relate the scientific evidence to the knowl¬ 
edge that college students have at their mental fingertips. 

Cowan points out that there are both time and capac¬ 
ity limits. Figure 10.2 suggests that what we are conscious 
of at any moment - the focus of attention-may be capac¬ 
ity-limited. In contrast, the 'active memory' in Figure 10.2 
may be time-limited; it fades over time. As you focus on the 
sentence you are reading right now, for example, you will 
notice that it fades from memory in a few seconds. 

Cowan suggests that the limited-capacity system may 
in fact run into both kinds of limits, depending upon the 
degree of attentional focus. If our mental focus zooms 
into an array of information, only a single item may 
become conscious; but if our mind's eye zooms out, up to 
four different items can be apprehended. We know that 
something like that is true for vision. Visual attention can 
zoom in and out of the details of the optical array, and we 
have seen how in the brain, the control of visual atten¬ 
tion and eye movement involves the overlapping regions 
of the cortex (Chapter 8). Thus visual attention and eye 
movement control may have co-evolved over vertebrate 
evolution. A similar covert 'attentional zoom capacity' 
may have evolved along with the capacity to zoom into a 
physical light display. That is at least a testable hypothesis 


with the brain imaging tools we have today (Cowan, 
2008). 

Some questions are still debated. For example, Oberauer 
and Bialkova (2009) suggests that there is a conscious focus 
of attention that holds only one item, along with a 'fringe' 
that holds the three to five items. But there is no disagree¬ 
ment that the capacity numbers are very low. 

The narrow bottlenecks in our mental information 
flow have a profound effects on learning and problem 
solving. People differ in their ability to guide and con¬ 
trol their attention, to avoid distraction, and to persist 
in attending to a difficult task. Those differences corre¬ 
late with individual cognitive traits, like standard intel¬ 
ligence, persistence, and processing reaction time. They 
also develop in predictable ways over the lifetime. 

Halford, Cowan, and Andrews (2007) have proposed 
a specific relationship between working memory and 
reasoning, such as 'Knights and Knaves' puzzles (2007; 
http:/ / en.wikipedia.org/wiki/Knights_and_Knaves). 
Knights and Knaves puzzles can be solved by binding 
elements (A and B) into slots (the knights and knaves). 

Capacity limits in both working memory and reason¬ 
ing can be attributed to the number of bindings to slots in 
a coordinate system or relation. Working memory is lim¬ 
ited to approximately four items that can be kept active, 
whereas representations in reasoning are limited to four 
interrelated variables. 
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1.0 WORKING MEMORY 

In a broad sense of the term, working memory is the 
domain of problem solving, language, and thought 
(Chapter 2). It is The set of mental processes hold¬ 
ing limited information in a temporarily accessible 
state in service of cognition' (Cowan et al., 2005). We 
need working memory to perform mental arithme¬ 
tic, to carry on a conversation, or to solve a pathfind¬ 
ing problem - How do I walk home from here? You 
cannot understand the sentence you are reading 
now without keeping words, ideas, and syntax in 


immediate memory. When we think about a prob¬ 
lem, we constantly use inner speech and visuospatial 
thinking, directing attention to what is currently most 
important. As discussed in Chapter 8, attention leads 
us to become conscious of sensory events, of inner 
speech and action planning. Finally, whatever we are 
actively thinking about interacts with what we already 
know - our long-term store of memories, knowledge, 
and skills. Every word in this sentence is also part of 
our long-term vocabulary. Every eye movement you 
make is based on long-practiced routines (see Chapter 
9). Thus, all the components of Figure 10.3 come into 
play in the realm of thinking and problem solving. 
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A functional framework 
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FIGURE 10.3 Problem solving in the functional diagram. Working memory (WM) is constantly involved 
in problem solving. However, WM functions also make use of stored information, such as the words and mean¬ 
ings of natural language, habits and motor skills, and various types of memory, shown in the gray boxes along 
the bottom of the diagram. 


Surprisingly, much of this chapter is not about the 
colored boxes in the functional diagram, but about 
the row of gray boxes along the bottom. The colored 
boxes refer to active processes, the ones that require 
neuronal firing and integration. But they constantly 
run along habitual pathways encoded by previous 
active processes, which have now formed permanent 
networks of connections. Such permanent stores may 
not show up directly in brain imaging studies because 
they are encoded in the connective strengths between 
neurons. Methods like functional magnetic resonance 
imaging (fMRI) activity may therefore under-repre¬ 
sent the vast amount of long-term knowledge. 


1.1 Working memory overlaps with attention, 
conscious events, and episodic recall 

Experiments generally aim to distinguish between 
working memory and related tasks. For example, in 
some working memory experiments, activity from sin¬ 
gle neurons in prefrontal cortex is recorded from cats 
or monkeys using a 'delayed match to sample' experi¬ 
ment in which the animal is given a stimulus, then 
must wait during a delay period after the stimulus 


disappears, and then point to the remembered stimu¬ 
lus among several others. Typical findings are that neu¬ 
rons keep firing during the delay portion, interpreted 
as an example of working memory function in which 
the animal must keep in mind the stimulus while wait¬ 
ing to perform a learned task (Goldman-Rakic, 1995; 
see Chapter 9). Selective attention presents people with 
two competing inputs. Conscious cognition compares 
conscious (reportable) to unconscious (unreportable) 
stimuli, using experimental methods like binocular 
rivalry (see Chapters 6 and 8). And episodic memory 
retrieval may involve asking people about their mem¬ 
ory of yesterday's lunch. By using precise comparison 
conditions, we often find different parts of the brain 
'lighting up'. New and reliable insights often come 
from these kinds of studies. 

There is another side, shown in Figures 10.4 and 
10.5, demonstrating widespread activation of frontal 
and parietal regions for four different brain activi¬ 
ties that we often separate from each other: working 
memory, attention, episodic recall, and conscious per¬ 
ception. The overlap is enormous. That does not mean 
that they are identical theoretical constructs. They can 
be distinguished experimentally. But we cannot forget 
that the brain does whatever it does, not necessarily 
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FIGURE 10.4 Proposed working memory regions. Common 
regions involved in working memory functions. Source : Schneider 
and Chein, 2003. 

what we design our experiments to do. In this chap¬ 
ter, we will use the term 'working memory' to explore 
language and thought, but we might equally talk 
about it in terms of conscious and unconcious cogni¬ 
tion, perception, and long-term memory The most 
active aspects of working memory tasks are conscious 
or voluntary It has been suggested therefore that pas¬ 
sive working memory functions may be controlled by 
conscious/voluntary features, like perceptual input, 
recall, rehearsal, action planning, and response output 
(Baars and Franklin, 2003). 

The limits of working memory must have posed 
difficult challenges for humans before the invention 
and spread of writing. Today we can scribble notes on 
a piece of paper, or look up information on the web. 
But over the neolithic period no one could write down 
their shopping list before going hunting and gather¬ 
ing. Perhaps for that reason, spoken language gives us 
many tricks and tools for chunking large amounts of 
information into smaller packages: sentences, words, 
phrases, sayings, narratives, personal names, and 
pronouns all serve as pointers and reminders (Figure 
10.6). We also condense large amounts of knowledge 
by means of named abstractions and classification 
schemes (Box 10.1). Much of our large vocabulary 
consists of such labels. The vocabulary of natural lan¬ 
guage is a treasury of culturally compounded chunks 
of meaningful information (see Chapter 11). 

Each chunk may point to a very large body of 
knowledge. In this book, the word 'brain' can be thou¬ 
ght of as a link to everything you know about cogni¬ 
tive neuroscience. By pointing to knowledge via brief 
labels, we can optimize our cognitive bottlenecks. 


Right 


(a) Attention 






(d) Conscious perception 

FIGURE 10.5 Overlapping brain regions support working 
memory, selective attention, autobiographical retrieval, and conscious 
perception. This figure shows schematically the widespread activa¬ 
tion of frontal and parietal regions for four different brain activities 
that we often separate from each other: working memory, attention, 
episodic recall, and conscious perception. There is substantial over¬ 
lap in these regions and it is not obvious that they can be separated. 
Attention = simultaneous selection, working memory = delayed 
selection, conscious perception = comparing a seen target to the 
same unconscious target, episodic retrieval = becoming conscious of 
an autobiographical event retrieved from memory While these four 
brain activities are typically separately tested in experiments, there 
are many overlapping features of those functions. Source : Naghavi 
and Nyberg, 2005. 

2.0 EXPLICIT PROBLEM SOLVING 

Unterrainer and Owen (2006) described the basic con¬ 
ditions of explicit problem solving: 

First, one needs to create a mental representation of both 
the current situation and the goal. Furthermore, these 
representations have to be linked by establishing which 
actions are needed to transform the current state into the 
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FIGURE 10.6 Some frontal capaci¬ 
ties for high-level thinking. The frontal 
lobes are also needed for advanced skills, 
ranging from the ability to understand 
other people's intentions to reasoning, 
imagination, and self-understanding. 
These frontal capacities constantly inter¬ 
act with posterior and subcortical brain 
regions. Source : Wicker et al., 2003. 


goal state. Problems therefore have three general char¬ 
acteristics: (1) an initial state, or the state in which the 
problem solver sorts out the givens; (2) a goal state, or 
the solution state that the problem solver tries to achieve; 
and (3) the steps that the problem solver takes to trans¬ 
form the initial state into the goal state that initially may 
not be obvious (Sternberg and Ben-Zeev, 2001). 

Problem-solving can be thought of as finding a path 
through a maze of choice-points between possible sub¬ 
goals toward a final goal. In the standard puzzle called 
the Towers of Hanoi, we are given a problem as in 
Figure 10.8. The goal is to end up with the three disks 
on the rightmost rod in the same order they are shown 
at the beginning. This requires subgoal planning. 
The tree of choices is called a problem space, as in Figure 
10.9. Some sequences of steps lead to a solution while 
the others do not. Figure 10.10 shows brain regions 
active while solving Tower of Hanoi puzzles, notably 
the dorsolateral prefrontal cortex (DL-PFC). 

Towers problems are puzzles rather than real-life 
problems. Real problems for human beings are such 
things as how to travel from one place to another across 


difficult and dangerous territory, how to find food and 
shelter, how to protect children and group members 
from dangers, how to maintain relationships, or how 
to prevail in a competitive situation. Life problems 
are rarely easily defined, and the stakes can be much 
higher. Nevertheless, even simple puzzles can repre¬ 
sent important features of real problems. Ideas like 
goals and subgoals, choice points in a problem space, 
and costs and benefits show up in many kinds of ways. 
While a complete problem space description is rarely 
available for more real-life problems, a useful strategy 
is to break down larger problem spaces into subgoals 
that can be described explicitly and completely. 

2 A Executive control in problem solving 

The cliche that 'generals tend to wage the previous 
war' applies to many kinds of problems. In science, 
for example, there is a tendency, even among the 
most celebrated scientists, to become focused on one 
major approach which may not always work. One 
famous case is Albert Einstein, who had an intense 
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BOX 10.1 How chess experts chunk known game positions 


Expert memory for chess position 

Much of what is known about expertise goes back to De 
Groot (1946) a and Chase and Simon (1973) b . One of De 
Groot's enduring contributions was to demonstrate the 
existence of clear differences between levels of player in a 
memory task, involving the brief presentation of a position 
taken from a tournament game. Typically, players at and 
above master levels recall the entire position almost per¬ 
fectly, but weaker players perform poorly (see Figure 10.7). 

However, Chase and Simon found no difference in recall 
of random positions between their three subjects: a master, 
a class A player, and a novice using their model CHREST 
(Chunk Hierarchy and REtrieval STructures). This uni¬ 
formly poor recall of random positions taken together with 
the superior performance of masters and grandmasters on 
game positions presented such a vivid illustration of the 
principle that knowledge is the key to expertise that it has 
become a classic finding, widely cited in textbooks of cog¬ 
nitive psychology and in papers on expertise. 

However, an earlier version of CHREST, a re¬ 
implementation and extension of MAPP (Memory-Aided 
Pattern Recognizer, Simon and Gilmartin, 1973) c , made 
contrary predictions about the recall of random positions. 
In the chess simulations, CHREST is trained from a data¬ 
base of master games, identifying patterns of pieces in 
these positions. As expected, the model's ability to remem¬ 
ber game positions improved as the number and average 
size of its chunks increased. However, the model also 
showed a small, but robust increase in recall with random 
positions. The skill differences in recall were the result of 
an easily explained mechanism: simply by chance, a larger 
discrimination network is likely to include patterns found 
in random positions. A systematic review of experiments 
that asked chess players to recall random positions 01 yields 
12 studies in which masters demonstrated some superior¬ 
ity, and only one. Chase and Simon's study b , where the 
master actually did worse than novices. Although the skill 
differences were not significant in most studies because 
of lack of statistical power, the effect becomes clear when 
the various studies are pooled (see Figure 10.7(b)). The 
fact that perceptual chunking provides masters with an 
advantage even in random positions offers strong sup¬ 
port for chunk-based theories, and is hard to explain for 
theories of expertise based upon high-level knowledge or 
schemata 3 . For a review, see Gobet, 1998. 

References 

a De Groot, A. D. (1946). Het Denken van den Schaker. Noord 

Hollandsche. 

b Chase, W. G. & Simon, H. A. (1973). Skill in chess. American 

Scientist, 62, 393MD3. 



(a) 



FIGURE 10.7 (a) Types of positions typically used in 

chess memory research. A game position taken from a mas¬ 
ters' game (left), and a random position obtained by shuffling 
the piece locations of a game position (right), (b) Mean number 
(averaged over 13 studies) of pieces placed correctly as a func¬ 
tion of position type (game or random) and skill level. Positions 
had 25 pieces on average, and the presentation time was <10 s. 
Error bars indicate standard errors of the means. Source : Adapted 
from Gomet and Simon, 1966. 

c Simon, H. A., & Gilmartin, K. J. (1973). A simulation of mem¬ 
ory for chess positions. Cognition Psychology, 5, 29-46. 
d Gobet, F., & Simon, H. A. (1966). Recall of rapidly presented 
random chess positions is a function of skill. Psychonomic 
Bulletin & Review, 3, 159-163. 

e Gobet, F. (1998). Expert memory: a comparison of four theo¬ 
ries. Cognition, 66, 115-152. 


dislike of quantum mechanics, even though it made 
it possible to explain phenomena that could not be 
addressed any other way. Einstein was philosophi¬ 
cally deeply opposed to the idea that physics could 
be statistical rather than deterministic, and tried for 
30 years to find an alternative to quantum theory. 


Hundreds of other examples can be found in the 
history of science. 'Fixedness' appears to be a very 
widespread difficulty in human problem solving. 

It is easy to induce functional fixedness in problem 
solving (Duncker, 1945). One only needs to encourage 
subjects to become accustomed to one way of reaching 
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FIGURE 10.8 The Towers of Hanoi. Shallice (1982) used the puzzle called the Towers of London (also called Towers of Hanoi) to diagnose 
frontal impairments. The task is to move the rings one at a time to match a target scheme. The difficulty level of these puzzles can be adjusted, as 
in the figure on the right side. Tower problems have been standardized for clinical testing, and have led to a great deal of research. Instructions 
on effective strategies improve performance for people with intact frontal lobe functioning. Source : Miller and Wallis, in Squire et al. r 2003. 





FIGURE 10.9 A problem space. The problem space for the 
towers puzzle shows many possible positions, choice-points, and 
pathways. 


the goal, even when that way no longer works. Func¬ 
tional fixedness can be shown in a great range of tasks. 
In language there are 'garden-path sentences' that mis¬ 
lead readers into misunderstanding the syntax of a 
sentence. A famous example is the sentence. The horse 
raced past the barn fell'. This grammatically correct 
sentence can cause many minutes of puzzlement. The 
problem comes from the fact that there is an outer clause. 



FIGURE 10.10 Dorsolateral prefrontal cortex in tower prob¬ 
lems. Tower problems elicit brain activity in the middle part of the 
dorsolateral prefrontal cortex (DL-PFC), a crucial region for execu¬ 
tive functions. Source : Unterrainer and Owen, 2006. 


'The horse ... fell' and an inner clause, 'raced past 
the barn'. The sentence can be rephrased as, 'The horse 
which raced past the barn fell'. However, in English the 
marker 'which' is optional for subordinate clauses. 

The sentence therefore leads the reader down the 
wrong path, and we only discover at the end of the 
sentence that we are left with an extra verb, 'fell'. 
Such misleading sentences are far more common 
than we realize, simply because we normally interpret 
language based on contextual knowledge. For example. 
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FIGURE 10.11 The Wisconsin Card Sorting Task. The WCST 
encourages subjects to adopt a certain rule like The yellow color 
predicts correct cards'. At some point a different rule comes into 
play, such as number or shape of items. This is a challenge to our 
ability to think of alternatives to the first rule, and people with fron¬ 
tal lobe impairments will typically perform poorly when the rule is 
shifted. Source : Grant, 1993. 


we might be watching horses racing around a stud 
farm, and know that one horse raced past the barn, 
while another one raced past the farmhouse. 

Fixed and misleading mental sets are very com¬ 
mon. Figure 10.11 shows the Wisconsin Card Sorting 
Task (WCST), which is designed to induce a mislead¬ 
ing set in a simple card sorting task. Subjects are asked 
to guess which card is 'correct', using color, number, 
or shape. They are given feedback for each guess. 
Initially they are rewarded for one pattern, e.g. the 
rule that the color yellow is always correct. At some 
point the rule is changed without telling the subjects. 
The time and number of missed trials needed for sub¬ 
jects to shift set is taken as an index of their ability to 
test different hypotheses. 

Patients with impaired frontal lobes often lack cog¬ 
nitive flexibility, and tend to score low on the WCST. 
Perseveration in unsuccessful strategies is a marker of 
disorders like Alzheimer's dementia (Ridderinkhof 
et ah, 2002). The WCST is especially useful when fron¬ 
tal lobe damage is too subtle to be detected by stand¬ 
ard brain scans. 

Changing rules is difficult when we are mentally 
fatigued or drowsy or otherwise impaired. Even switch¬ 
ing from one task to another seems to require additional 
mental resources beyond those involved in routine and 
automatic actions (Figure 10.12). Thus, drivers who are 
sleepy might find it harder to make fast decisions in 
unpredictable traffic situations, presumably because 
their frontal lobe may be functioning below par. Workers 



FIGURE 10.12 Task switching. Regions of high activity during 
task switching overlap with brain areas required in other executive 
tasks. Compare to Figure 10.19. (page 59). Source : Braver et al, 2003. 


Activity monitor Attention 



FIGURE 10.13 A proposed model. On the outer surface of 
each hemisphere, peak activity during problem solving appears in 
the dorsolateral prefrontal cortex (DL-PFC). During task conflict 
or errors we find high activity in the forward part of the cingulate 
cortex (ACC). These two territories consistently show high activity 
when a cognitive task is difficult. Source: Schneider and Chein, 2003. 


on night shifts have been shown to be more error prone 
than daytime workers, and frontal lobe function may 
have something to do with that difference as well. 

Figure 10.13 shows a model of executive function¬ 
ing from Schneider and Chein (2003). We will use it to 
summarize our current knowledge about explicit prob¬ 
lem solving. The area marked DL-PFC (dorsolateral 
prefrontal cortex) is considered to be a goal proces¬ 
sor. Thus, in the Towers of Hanoi, goals and sub goals 
would be represented in this region. The ACC 
(anterior cingulate cortex) monitors errors and conflicts 
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between goals (see Chapters 8, 11, and 12). PPC (the 
posterior parietal cortex) is involved in attentional 
control (see Chapter 8), while the medial temporal 
cortex (MTL) is the gateway to episodic memory, the 
autobiographical store of past conscious experiences 
(Chapter 9). Finally, the thalamus (marked THAL) is 
considered to be a 'gating and reporting relay'. The 
last label may be controversial, and really stands for a 
broader thalamo-cortical gating and reporting system 
(see Chapters 5 and 8). 

We have already seen the DL-PFC involved in the 
Towers of Hanoi problem and in task shifting (above). 
Figure 10.14 shows how both the DL-PFC and ACC 
become involved in five different tasks requiring 
executive effort, including response conflict (as in the 
Stroop task), novelty, immediate memory loading, and 
even perceptual difficulty. 

Figure 10.15 summarizes ACC activation across 
yet different executive tasks, including emotion¬ 
ally demanding ones. As we might expect, executive 
regions become involved in a very wide range of con¬ 
ditions that require flexible decision making. In con¬ 
trast, as we will see, unconscious memory traces and 
skills do not recruit these executive regions. Instead, 
they are thought to be encoded in widespread traces 


in the temporal cortex and elsewhere in the brain 
(Moscovitch et ah, 2006). 

This model therefore shows general-purpose func¬ 
tions, like: 

1 sustained goal pursuit and deciding on subgoals 

2 maintaining attentional focus on the task 

3 inhibitory control over distracting thoughts and 
emotions 

4 metacognitive monitoring of the quality of 
required sensory, motor, language, and immediate 
memory functions. 

We will later see how executive processes also make 
constant use of stored information, including chunked 
information that is needed to circumvent capacity 
limits. 

3*0 MENTAL WORKLOAD AND 
CORTICAL ACTIVITY 

Effortful tasks show a wider spread of brain activity, 
even beyond the executive regions of the frontal cor¬ 
tex. For example, in a classic study. Smith and Jonides 
(1998) found dramatically expanded cortical activity 



FIGURE 10.14 Mental effort activates executive regions. Left panel: Duncan and Owen (2000) showed that five very dif¬ 
ferent tasks that all involve mental effort activate lateral and medial sides of the frontal lobe. The five tasks shown are response 
conflict (green dots), task novelty (pink), the number of elements in working memory (yellow), the delay required in working 
memory (red), and perceptual difficulty, such as visually obscured stimuli (blue). Abbreviations: CC: corpus callosum; IFS, infe¬ 
rior frontal sulcus; SF: Sylvian fissure. The image on the right shows a different perspective of the ACC (anterior cingulate cor¬ 
tex) and DL-PFC (dorsolateral prefrontal cortex) (MacLeod and MacDonald, 2003). Source: Left : Duncan and Owen, 2000; right: 
MacLeod and MacDonald, 2003. 
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FIGURE 10.15 Error detection and resolution. 
As in Ligure 10.12, this one shows a number of dif¬ 
ferent tasks, all of which activate parts of the anterior 
cingulate cortex (ACC). Previous studies showed that 
conflict monitoring (as in the Stroop effect) evoked 
ACC activity. However, tasks as different as ambigu¬ 
ous emotion picture; induced anxiety; expected, unex¬ 
pected, and experienced pain; induced emotion; and 
expected negative emotional pictures, all showed 
reliable ACC activation. Other studies locate the emo¬ 
tional aspects of ACC activation to the anterior tip of 
the ACC, and cognitive activation, as in error detec¬ 
tion, to the upper part of the ACC. Source : Botvinick 
et al, 2004. 




FIGURE 10.16 Loading working memory: the n-back task. 
In the 0-back case, subjects only need to report what they see. As 
n goes from 0-back to 3-back, they must keep in mind more items, 
keep track of the order of each one, and at the same time notice each 
new slide to add to their mental stack. Task difficulty rises steeply 
with the number of items to be kept in running memory 


as a function of memory load. These authors used an 
'n-back' technique (Figure 10.16). Subjects were shown 
a series of memory items. In the 1-back condition, they 
only had to recall the last item; in the 2-back case, the 
item shown before the last one; in the 3-back case, 
the item shown 3 slides before. The n-back task is 
the human equivalent of a hamster's running wheel. 


One must keep constantly working to keep up with 
the flow of information. Difficulty level rises very 
quickly with the number of items to be kept in mind. 
Results from the Smith and Jonides (1998) study are 
presented in Figure 10.17. 

What we cannot see yet, even with advanced brain 
recording methods, is the strength of connections 
between brain cells. Figure 10.18 indicates that men¬ 
tal effort also changes connection strengths, the neural 
signaling density between cortical locations. 

Judging by brain activity, cognitive effort is one 
of the biggest factors in brain functioning. That may 
seem odd. After all, tasks that take mental effort are 
not usually the most complex or sophisticated ones 
we perform. The sentence you are reading right now 
is far more complex than the n-back memory task we 
just discussed. Why doesn't the sophisticated lan¬ 
guage processing use a great deal of brain activity? 
Figure 10.19 suggests an answer: the level of activity 
in cortex (at least) drops with practice and automatic- 
ity. While language analysis is extremely complex, it 
is also highly practiced over a period of years. Thus, 
neural firing or brain metabolism is not a direct 
measure of the complexity of some mental process. 
Rather, it seems to indicate the recruitment of neuronal 
resources that are needed to work together to perform 
a task that is new or unpredictable. Once even very 
complex processes are learned, they seem to require 
less cortical activity. Automaticity also involves a loss 
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FIGURE 10.17 Effortful tasks recruit wider brain regions 
compared to a no-memory control. Notice that increased working 
memory load recruits progressively wider regions, including parts 
of frontal and posterior cortex and cerebellum. Thalamus and basal 
ganglia are not shown, but are likely to be more mobilized as well. 
Source : Smith and Jonides, 1997, Figure 10. 


of conscious access and voluntary control, as assessed 
by behavioral measures (Schneider, 1995; Baars, 2002). 


4*0 USING EXISTING KNOWLEDGE 

One theme of this chapter is how the human brain 
makes constant use of chunking strategies to solve 
problems within the capacity limits of immediate 
memory How does the brain manage this? 

One way to think about chunking is to reverse our 
usual concept of working memory and emphasize the 
degree to which mental operations make use of existing 
learning, encoded in forests of acquired synaptic con¬ 
nections (see Chapter 3). Working memory is crucially 
dependent upon stored long-term information. Your 
understanding of this sentence depends upon your 
memory for words, sentence structure, and meaning. 
Cowan (2001) and others therefore suggest that we can 
flip the roles of working and long-term memory, and 
conceive of working memory as playing upon years of 
previous stored information (Figure 10.2). 

Current brain imaging methods reveal active neu¬ 
ronal signaling, but not the synaptic connectivities 
that encode those years of learning. Brain recording 
is therefore much like watching the flow of traffic in 
a city from outer space. We might easily see car lights 
streaming back and forth each day. But with our cur¬ 
rent instruments it is harder to observe the slower 
process of building new roads, highways, and park¬ 
ing lots that make it possible for traffic to move. Yet 
the entire system depends upon the physical layout of 
streets and highways, a kind of permanent memory of 
the city. A space observer could learn a lot from traffic 
patterns - one neighborhood may be a financial center, 
another one might have an airport. But, on weekends. 


1-BACK 



2-BACK 



FIGURE 10.18 Connection strengths change 
with task difficulty. The colored lines in the two brain 
diagrams indicate connection strengths between 
executive and other regions of the brain. Connection 
patterns change with increase in memory workload. 
Source : Honey et al, 2002. 
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FIGURE 10.19 Executive activity drops with practice. 
After practice in predictable tasks, cortical activity is dramati¬ 
cally reduced. This summary image shows reductions in executive 
regions (compare to Figure 10.12). However, other active cortical 
regions also show decrements with increased practice and auto- 
maticity. It is believed that control over routine tasks is relegated to 
subcortical regions like the basal ganglia and cerebellum. However, 
intermittent cortical control may remain at unpredictable choice- 
points in the task. Source : Schneider and Chein, 2003. 


traffic flows might change. In bad weather more peo¬ 
ple may stay home. Depending upon unknown condi¬ 
tions traffic may flow along different routes, while still 
making use of the permanent connectivity of streets 
and highways. Just watching signal flow reveals only 
part of what makes the brain work. 

Thus, knowing the connections and their strengths - 
the streets and highways - is an essential part of the 
puzzle that is still hard to study. Yet we know that long¬ 
term memory is crucial (look back at Box 10.1). We will 
take a look at evidence for long-term changes next. 


4*1 Practice and training may change 
connectivities in the brain 

As discussed previously, long-term memories are 
believed to be encoded in the connections between 
neurons. If memories are encoded in synaptic connec¬ 
tions, they may not show up directly in brain measures 
like fMRI. Connections and active brain processes are 
not necessarily correlated; in fact, in the section above. 
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FIGURE 10.20 Long-term brain changes in experts. Maguire 
et al. (2000) found that London taxi drivers showed larger posterior 
hippocampal volume on the right side than controls, consistent with 
the known role of hippocampus in spatial processing. However, this 
methodology does not prove that the number of synaptic connec¬ 
tions is greater as a result of expertise. The difference could be due 
to other factors, such as myelination, the number of support cells, 
and the size of neurons. However, the sheer size of relevant brain 
regions seems to relate to expertise in other studies as well. Source : 
Maguire et al ., 2000. 


we found evidence that more skilled and expert tasks 
might show less cortical activity. Thus, long-term 
stores may not be reflected in fMRI, magnetoencepha¬ 
lography (MEG), electroencephalography (EEG), and 
other standard measures. 

Other measures are currently being developed to 
assess the efficiency of synaptic connections in the 
brain's neural networks. One possibility is shown in 
Figure 10.20, from a classic study by Maguire et al. 
(2000) on London taxi drivers, who must pass exami¬ 
nations about their knowledge of the complex street 
map of London. After years of experience in a real- 
world spatial task like this, would their brains show 
differences as a result? 


4*2 Semantic memory 

We have previously noticed the close connection 
between sensory and motor systems in the brain, and 
their use for endogenous or 'inner' cognitive functions. 
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It seems that visual imagery makes use of visual cor¬ 
tex, and that inner speech makes use of speech cortex. 
Figure 10.20 may take that rule of thumb a step farther, 
in that semantic concepts seem associated with the 
temporal lobe (including the medial temporal region 
and the hippocampal neighborhood, not visible in 
the figure). More generic concepts are believed to be 
encoded posterior to unique concepts in the left lateral 
temporal lobe. Biological motion - the rather sinuous 
movements of animals - is located near area MT (the 
visual motion region), while human-made artifacts 
and instruments seem to activate a slightly different 
part of the occipital-temporal area. There is indepen¬ 
dent evidence that tools may also activate regions close 
to somatosensory and motor cortex. Hauk et al. (2004) 
recently found fMRI evidence that, 'action words refer¬ 
ring to face, arm, or leg actions (e.g. to lick, pick, or 
kick) . . . differentially activated areas along the motor 
strip that either were directly adjacent to or over¬ 
lapped with areas activated by actual movement of 
the tongue, fingers, or feet'. This supports the general 
theme of sophisticated and biologically recent semantic 
capacities making use of long-established brain regions 
adapted to dealing with the sensorimotor world. 

In a recent review, Martin and Chao (2001) wrote: 

Distributed networks of discrete cortical regions are 
active during object processing. The distribution of 
these regions varies as a function of semantic cat¬ 
egory. The same regions are active, at least in part, 
when objects from a category are recognized, named, 
imagined, and when reading and answering questions 
about them.... 

Taken together, these data suggest that ventral 
occipitotemporal cortex may be best viewed not as a 


mosaic of discrete category-specific areas, but rather 
as a lumpy feature-space, representing stored informa¬ 
tion about features of object form shared by members 
of a category. .. . 

A feature-based model can accommodate the 
observation that an arbitrary category such as chairs 
elicited a pattern of neural activity distinct from other 
object categories (i.e. faces and houses). Clearly, it 
would be difficult, as well as unwise, to argue that 
there is a 'chair area' in the brain. There are simply 
too many categories, and too little neural space to 
accommodate discrete, category-specific modules for 
every category. In fact, there is no limit on the number 
of object categories. Feature-based models can pro¬ 
vide the flexibility needed to represent an infinite 
variety of objects. 

In other words, the temporal-prefrontal system 
shown in Figure 10.21 is not likely to be a dictionary 
of all the semantic categories we know. There are too 
many of them. Rather, it may represent important fea¬ 
tures of major categories, which can index a number 
of categories represented in widely distributed cortical 
networks. We can find parts of this large region that 
respond more to houses than faces or more to tools 
than houses or cars. The 'tool'-biased regions might 
index a vast range of objects that can be used as tools - 
not just hammers and chisels, but perhaps computers 
and telephone companies. It would be very interest¬ 
ing to see whether one category (such as human faces) 
would show a Tool use' bias if it were stated as a tool. 
For example, masks of human faces can be 'used as a 
tool' to scare people on Halloween. Such experiments 
might test the hypothesis that 'tool' regions might, in 
fact, represent a more abstract proposition such as 'can 
be used to accomplish a concrete goal'. 



FIGURE 10.21 Brain regions involved 
in semantic memory. A recent summary of 
semantic memory location in the left hemi¬ 
sphere (figure on the right) and the bottom of 
the temporal lobe, facing upward (on the left). 
The spatial location of the bottom of the tem¬ 
poral lobe may take some study to understand 
clearly. Notice that there are believed to be 
semantic gradients between specific visual object 
areas and more abstract visual forms. Semantic 
working memory involves constantly looping 
activity between temporal and frontal regions, 
which must make use of subcortical connec¬ 
tions running between them. Source : Martin and 
Chao, 2001. 
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Neurologists have suggested that 'object concepts 
are defined by sensory and motor attributes and fea¬ 
tures acquired during experience'. This appears to be a 
powerful organizing principle for concepts in the brain. 

Possible contradictory evidence has come from 
single-cell studies in epileptic patients, with electrodes 
implanted in the temporal lobe. Some of these stud¬ 
ies have shown very specific categorical responses, for 
example, to very different photos of President Clinton 
(Kreiman et al. f 2002). Presumably, highly specific neu¬ 
ronal responding represents one node in a large net¬ 
work, or set of overlapping networks, having to do 
with political figures or famous people. It is possible 
that, as we learn to sample more locally in the tem¬ 
poral lobe, such networks containing highly specific 
information may be observed. 

As shown in Figure 10.21, prefrontal regions close 
to the classical Broca's area may support a 'semantic 
working memory system', somewhat separate from 
the phonological and visuospatial components of 
working memory. 


4.3 Abstract concepts, prototypes, and 
networks 

Intuitively, we tend to have some misleading ideas 
about human cognition. One is that we carry pic¬ 
tures in our heads that represent the perceptual world 
around us. The evidence suggests instead that we 
tend to use visual images that are prototypical remind¬ 
ers of categories in the world, rather than accurate 
images of categories like chairs and movie stars. We 
seem to have a network of perceptual, cognitive, and 
motoric knowledge about chairs and their uses. Such 
networks can be accessed by prototypical pictures 
of chairs (Figure 10.22). Humans have a preference 
for such prototypical images, but they are not accu¬ 
rate depictions of all the chairs we have ever known 
(Rosch, 1975; Barsalou, 1999). Rather, they are special 
members of a category - of chairs or movie stars - 
which stand for the entire category. The wooden chair 
in Figure 10.22 may not be the average chair you have 
seen and used. But for the tested group that kind of 
visual image would come to mind more easily than 
the plastic or metal chairs we tend to see more often. 

Barsalou (1999, 2005) has suggested that humans 
have a strong perceptual bias, even in dealing with 
abstract categories. One reason is that we do not come 
into the world with an understanding of abstractions. 
The early years of childhood are devoted to sensory, 
motor, and emotional exploration which, from an 


evolutionary point of view, are matters of survival. 
The developing brain may select early populations of 
neurons and connections before we learn adult con¬ 
cepts of the world. 




00 0 ) 

FIGURE 10.22 A set of visual prototypes tend to stand for 
more abstract categories. Source : Laeng et al., 2003. 
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FIGURE 10.23 Amodai versus perceptual symbol systems. Barsalou (2003) has suggested that humans think by way of "perceptual sym¬ 
bol systems' (a) rather than abstract categories with feature lists, (b) Such a proposal is consistent with the general tendency in the brain to 
build on what information it is given, especially early in life. Source : Barsalou, 2003. 


The same point comes from studies by Barsalou 
and others on 'perceptual symbol systems'. Do we 
represent the car in Figure 10.23 the way an engineer 
might, as having a list of features like engines and 
wheels? The evidence appears to show instead that 
the abstract concept of a car is more dependent upon 
perceptual features of the appearance of the car, and a 
cognitive re-enactment of the actions of cars (Barsalou 
et al., 2003). 

4.4 Knowledge comes in networks 

Another common intuition is that common words and 
concepts are simple because they often have simple 
names like 'car', 'brain', and 'person'. This intuition is 
also very misleading. Mental representations, includ¬ 
ing words, visual images, and concepts, should be 
viewed in terms of elaborate networks of knowledge. 
The cognitive evidence for such networks was exten¬ 
sive even before the advent of brain imaging meth¬ 
ods. It can indeed be found in the study of science in 
the mid-20th century (Kuhn, 1962). Naively, we tend 
to think of basic scientific ideas like 'gravity' and 
'molecule' as single ideas. Some philosophers therefore 
proposed that there must be a one-to-one relationship 
between the Newtonian concept of force and the phys¬ 
ical observations that gave rise to that concept. But 


others pointed out that there is no observable correlate 
of Newtonian force at all - we only measure mass and 
acceleration. 'Force' is a purely inferential concept in 
physics (see Chapter 1). Newtonian theory is not just 
a list of concepts, but a network of carefully defined 
ideas supported by standard experiments, by predict¬ 
able observations, and tied into a web of mathematical 
inferences. Scientific theories are therefore semantic 
networks, not just labeled collections of observations. 
The same argument applies to the words of natural 
language. 

These points are fundamental for understanding 
how the brain represents knowledge. In brain imag¬ 
ing we do not see abstract classes of objects. Rather, 
we see perceptual objects in sensory regions of the 
brain, which gradually shade into more abstract forms 
of representation (see Figure 10.21). Ideas appear to be 
represented in the cortex in terms of complex webs of 
learned connectivities, rather than localized filing sys¬ 
tems with neatly arranged conceptual categories. The 
brain is a very practical organ, always close to the sen¬ 
sorimotor and motivational world, rather than being 
an abstract logic machine. (The brain can do logic, 
of course, but most of the time it is focused on more 
down-to-earth processes.) 

The need for semantic networks, not simple concept 
nodes, is important to understand when we look for the 
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FIGURE 10.24 A proposed network 
for a single word. Each word in our lexi¬ 
con refers to a network of related perceptual, 
semantic, motor, and lexical nodes. Source : 
Lokendra Shastri, with permission. 


neural basis of cognitive units like words, ideas, and 
images. For example. Figure 10.24 shows how a single 
personal name might be represented in a neural net. 

Since related concepts often share features, the 
same line of reasoning suggests that feature-related 
neurons might participate in more than one concept 
network. Figure 10.25 illustrates this hypothesis for 
the concepts of 'tiger' and 'elephant'. 

It has been extremely difficult to discover specific 
cortical locations for specific concepts. Some of the 
exceptions are shown in Figure 10.26. 

4.5 Conceptual deficits 

In general, Martin and Chao (2001) report that: 

Patients with damage to the left prefrontal cor¬ 
tex (LPC) often have difficulty retrieving words in 
response to specific cues (e.g. words beginning with 
a specific letter, the names of objects belonging to a 
specific semantic category). . . . This suggests that 
the LPC plays a general, albeit crucial, role in retriev¬ 
ing lexical and semantic information. Patients with 
damage to the temporal lobes often have difficulty 
naming objects and retrieving information about object- 
specific characteristics. . . . This suggests that object- 
specific information may be stored, at least in part, in 
the temporal lobes. 

However, some language deficits seem to be very 
precise and limited. For example, in the bottom of 



FIGURE 10.25 Overlapping semantic networks for two con¬ 
cepts. The brain may have feature-sensitive neurons that participate in 
more than one semantic network. Source: Hodges and Patterson, 1997. 


the temporal lobe, neighboring regions seem to be 
devoted to somewhat different concepts, like chairs, 
faces, and houses. Tools, animals, and vehicles have 
been reported to show peak activity in separate areas. 
That does not mean that specific regions are the 
only ones involved in a concept like 'hammer' versus 
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FIGURE 10.26 Areas of increased activity for 
animals, vehicles, tools, and vegetables. Conceptual 
categories are hard to find in the brain. The excep¬ 
tions are very broad categories like the ones shown 
in this figure. Like other brain images, we can only 
see activation differences, between animals and 
tools, for example. The areas that show differen¬ 
tial conceptual activity are in the medial temporal 
region. Some scientists suggest that these specific 
categories may serve as general indices for retriev¬ 
ing larger and more specific networks related to 
elephants and apples, rather than showing us 
the specific neural code for the concepts. Source : 
Caramazza and Mahon, 2003. 


'bicycle'. Rather, we see peak activity in different parts 
of the temporal lobe, and there are cases of very local 
deficits as well. 

Caramazza and co-workers have performed careful 
studies of a patient known as EW, who shows a deficit 
in naming animals (Caramazza and Shelton, 1998; Box 
10.2). By presenting pictures with known frequency 
and familiarity ratings, EW's knowledge of pictured 
categories could be tested in a controlled way. She 
had greater difficulty naming animals compared to 
other categories. EW also had difficulty simply recog¬ 
nizing animal pictures, but not other categories. Her 
deficit was not restricted to pictures, but also 
extended to spoken animal names. Yet the patient per¬ 
formed in the normal range on complex visual pic¬ 
ture processing, such as face recognition and picture 
matching. 

Finally, EW had trouble with major attributes of 
animals, but not attributes of other objects. Given 
questions like 'Does a whale fly'? and 'Does a cow 
have a mane'? she was incorrect about a third of 
the time. She performed better on other categories. 
While the idea of specific deficits is compelling, there 
are alternative interpretations of the evidence (Tyler 
and Moss, 2001). 


4*6 Judgments of quantity and number 

To the surprise of many, good evidence has emerged 
in recent years for a specific area of parietal cortex for 
number naming and judgments. Dehaene et al. (2004) 
suggest that the region shown in Figure 10.27 has a 
special affinity for quantitative judgments, though it 
may have other functions as well. When we perform 
mental arithmetic, however, as you might expect, 
inner speech areas are activated, presumably because 
we are mentally talking through the steps of subtrac¬ 
tion or multiplication (Figure 10.28). It would be inter¬ 
esting to find subjects who are skilled in using visual 
arithmetic, such as users of an abacus. We would not 
expect the phonological region to become activated 
for arithmetic operations in such people, but rather 
regions of the occipital and parietal cortex. 

5*0 IMPLICIT THINKING 

The story is told that as a child, Wolfgang Amadeus 
Mozart would tease his father Leopold (who was also 
his music teacher) by playing a phrase on the harpsi¬ 
chord and deliberately leaving it unfinished. Western 
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music almost always completes a series of notes or 
chords to end up where it started, at the tonic note of 
the scale. In Mozart's time that rule could not be vio¬ 
lated. Leopold Mozart would have to get out of bed, 
walk downstairs, and play the sequence of notes run¬ 
ning in his mind before he could go back to sleep. It 
was simply intolerable to hear an incomplete melodic 
series. You can get the same effect by playing a 
favorite recorded song, and stopping it just before it 
comes to a conclusion. 



FIGURE 10.27 A network for number judgments? This pari¬ 
etal network was found to be involved in number and quantitative 
judgments, especially the intraparietal sulcus. Source : Dehaene et al., 
2004. 


Completion of expected sequences is one example of 
implicit problem solving. We do not tell ourselves con¬ 
sciously to listen until the music comes to a resolution. 
But once started, we feel the need to do so. The goal is 
implicit or unconscious, like the rules of harmony and 
melody. Few people can explain those basic rules, but 
they have powerful effects nevertheless. If we compare 
listening to a song to solving the Tower of Hanoi puz¬ 
zle (above), music understanding is unconscious in 
almost all the ways the Tower problem is conscious. 

Most human problem solving is a mixture of 
explicit and implicit ingredients. We tend to underes¬ 
timate the complexity of implicit cognition, precisely 
because it is unconscious. In fact, it is our highly 
expert, overlearned habits that may be the most effi¬ 
cient tools for solving problems. The explicit aspect 
of problem solving may be better used for temporary 
executive functions in otherwise habitual tasks. For 
example, in driving a car we may be 'on automatic' 
much of the time, because of the routine and pre¬ 
dictable nature of the task. But when traffic becomes 
dense and unpredictable, when the car tire springs a 
leak, or when someone is distracting us by talking, 
executive control of driving may be more needed. 
Thus, there may be a flexible tradeoff between more 
controlled and more automatic aspects of the act of 
driving. The same principles may apply to other kinds 
of problem solving. 


5.1 Feelings of knowing 

Feelings of knowing and 'fringe' or 'vague' conscious 
events are believed to be very common (James, 1890). 
These include the well-studied 'tip-of-the-tongue' 


Left hemisphere 


Right hemisphere 



form 
Temporal lobe 


form 

Temporal lobe 


FIGURE 10.28 Mental arithmetic uses the phonological loop of inner speech. Notice the overlap in phono¬ 
logical tasks (such as mental rehearsal of a word list) and the subtraction task. Presumably inner speech is involved 
in subtraction, in this case. Source : Wynn in Ramachandran, 2002. 
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BOX 10.2 Specific conceptual deficits in patient EW 


An illustrative case of category-specific semantic 
deficit: patient EW 

To appreciate the remarkable nature of category-specific 
semantic deficits, consider the case of patient EW a . This 
patient presented with a disproportionate semantic 
impairment for the category 'animals' compared with 
other categories. Here we outline the empirical charac¬ 
teristics of EW's profile of impairment probably due to 
temporal lobe damage. 

Picture naming 

On subsets of the Snodgrass and Vanderwart (1992) b picture 
set matched jointly for familiarity and frequency, and for vis¬ 
ual complexity and familiarity, EW was disproportionately 
impaired at naming animals compared with naming non¬ 
animals (Table 10.1). This indicates that EW's category-specific 
deficit for picture naming cannot be attributed to uncon¬ 
trolled stimulus variables (e.g. Stewart et ah, 1992; Funnel & 
Sheridan, 1992; C/ d ). 

EW's picture-naming performance was not only quan¬ 
titatively different for animals and non-animals but was 
also qualitatively different. For animals, EW either named 
the picture incorrectly or did not recognize the picture, 
whereas for non-animals, EW recognized the picture but 
could not retrieve the name (Figure 10.29(a)). 

EW's naming deficit was restricted to the category 'ani¬ 
mals' and did not extend to the other living things such as 
'fruit/vegetables', for which performance was at ceiling. 

Sound identification 

EW was also impaired at naming animals compared with 
non-animals based on their characteristic sounds (8/32; 
25% correct versus 20/32; 63% correct: z 3.06, P < 0.05), 
indicating that the naming impairment is not restricted to 
visual input. 

Object decision 

EW was asked to decide ('yes' or 'no') whether a depicted 
object was real (see Figure 10.29(b) for examples of stimuli). 


TABLE 10.1 EW’s picture-naming performance for matched 
sets of items b 



Matched familiarity and 
frequency 

Matched visual complexity 
and familiarity 


Animals 

Non-animals 

Animals 

Non-animals 

EW 

12/22 (55%) 

18/22 (82%) 

7/17 (41%) 

16/17 (94%) 

Controls 

11/11 (100%) 

10.8 (98%) 

16.6/17 (98%) 

16/17 (94%) 

Range 

11 

10-11 

16-17 

16-17 


Performance on this task is interpreted as reflecting the 
integrity of the visual/structural description system (i.e. 
the modality-specific input system that stores representa¬ 
tions corresponding to the form or shape of objects, and 
which is used to access conceptual information). EW 
performed significantly below the normal range for dif¬ 
ferentiating real from unreal animals (36/60: 60% correct; 
control mean: 54/60: 90%) but within the normal range 
for differentiating real from unreal non-animals (55/60: 
92% correct; control mean: 50.5/60: 84% correct). 

Parts decision 

EW was asked to decide which of two heads (or parts) 
went with a headless body (or object missing a part) (see 
Figure 10.29(c) for examples of stimuli). EW was severely 
impaired on this task for animals (60% correct; normal 
mean = 100%) but performed within the normal range 
for artifacts (97% correct; normal mean = 97%). 

Visual processing 

EW performed within the normal range on complex 
visual processing tasks, such as visual matching and face 
recognition. These data indicate that EW does not have 
a general deficit for processing visually complex stimuli 



Kangaroo 1 Chicken 1 



(a) Examples of EW's 
naming errors 



(b) Stimuli from object-reality 
decision task 



FIGURE 10.29 (a) Examples of EW's naming errors, (b) Examples of stimuli from the object-reality decision task, (c) Examples of 

objects from the Tarts decision test' or 'Heads test'. 
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BOX 10.2 (Continued) 


and suggest that the impairment for object reality deci¬ 
sion for animals is categorically based. 

Central-attribute judgments 

EW was asked to decide whether a given attribute was 
true of a given item (see Table 10.2 for examples of stim¬ 
uli). EW was severely impaired for attributes pertaining to 
animals (65% correct; control range 85-100%) but within 
the normal range for attributes pertaining to non-animals 
(95% correct; control range 86-100%). EW was equiva¬ 
lently impaired for both visual/perceptual and func¬ 
tional/associative knowledge of living things (65% correct 
for both types of knowledge) but within the normal range 
for both types of knowledge for non-animals (visual/per¬ 
ceptual: 93.5% correct; control range 86-100%; functional/ 
associative: 98% correct; normal range 92-100%). EW's 
performance on answering central-attribute questions 
indicates that her deficit is not restricted to production. 

References 


TABLE 10.2 Examples of central-attribute questions 


Visual/Perceptual 

Functional/Associative 

Does a cow have a mane? 

Does a whale fly? 

Does a whale have a large tail fin? 

Does an eagle lay eggs? 

Does a whale have eight legs? 

Is a cow a farm animal? 


b Snodgrass, J., & Vanderwart, M. (1992). A standardized set 
of 260 pictures: norms for name agreement, familiarity, and 
visual complexity. Journal of Experimental Psychology. [Human 
Learning], 6,174-215. 

c Stewart, E, et al. (1992). Naming impairments following recov¬ 
ery from herpes simplex encephalitis. The Quarterly Journal of 
Experimental Psychobiology A , 44, 261-284. 
d Funnel, E., & Sheridan, J. S. (1992). Categories of knowledge? 
Unfamiliar aspects of living and nonliving things. Cognitive 
Neuropsychology, 9, 135-153. 


a Caramazza, A., & Shelton, J. R. (1998). Domain-specific 
knowledge systems in the brain: the animate-inanimate dis¬ 
tinction. Journal of Cognitive Neuroscience, 10, 1-34. 


BOX 10.3 Associative visual agnosia 


Case study of associative visual agnosia 

The subject was a 47-year-old man who had suffered an 
acute loss of blood pressure with resulting brain damage. 
His mental status and language abilities were normal, 
and his visual acuity was 20/30, with a right homony¬ 
mous hemianopia (blindness in the right visual hemi- 
field). His one severe impairment was an inability to 
recognize most visual stimuli. For the first 3 weeks in the 
hospital, the patient could not identify common objects 
presented visually and did not know what was on his 
plate until he tasted it. He identified objects immediately 
on touching them. 

When shown a stethoscope, he described it as 'a long 
cord with a round thing at the end' and asked if it could 
be a watch. He identified a can opener as a key. Asked 
to name a cigarette lighter, he said, 1 don't know' but 
named it after the examiner lit it. He said he was 'not 
sure' when shown a toothbrush. He was never able to 
describe or demonstrate the use of an object if he could 
not name it. If he misnamed an object, his demonstration 
of use would correspond to the mistaken identification. 
Identification improved very slightly when given the cat¬ 
egory of the object (e.g. something to eat) or when asked 
to point to a named object instead of being required to 
give the name. When told the correct name of an object, 
he usually responded with a quick nod and often said, 
'Yes, I see it now'. Then, often he could point out vari¬ 
ous parts of the previously unrecognized item as read¬ 
ily as a normal subject (e.g., the stem and bowl of a pipe 



FIGURE 10.30 Visual agnosia. A patient with associative 
visual agnosia was able to copy the everyday pictures above, but 
not to name them (Rubens and Benson, 1971). Source : Squire et al, 
2003. 
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and the laces, sole, and heel of a shoe). However, if asked 
by the examiner, 'Suppose I told you that the last object 
was not really a pipe, what would you say'? He would 
reply, T would take your word for it. Perhaps it's not a 
pipe'. Similar vacillation never occurred with tactilely or 
aurally identified objects. 

After he had spent 3 weeks on the ward, his object¬ 
naming ability improved so that he could name many 
common objects, but this was variable; he might cor¬ 
rectly name an object at one time and misname it later. 
Performance deteriorated severely when any part of the 
object was covered by the examiner. He could match 
identical objects but could not group objects by categories 
(clothing, food). He could draw the outlines of objects 
(key, spoon, etc.) that he could not identify. 

He was unable to recognize members of his fam¬ 
ily, the hospital staff, or even his own face in the mirror. 
Sometimes he had difficulty distinguishing a line draw¬ 
ing of an animal face from a man's face but always rec¬ 
ognized it as a face. The ability to recognize pictures of 


objects was impaired greatly, and after repeated testing 
he could name only 1 or 2 of 10 line drawings. He was 
always able to name geometrical forms (circle, square, tri¬ 
angle, cube). Remarkably, he could make excellent copies 
of line drawings and still fail to name the subject (Figure 
10.30). He easily matched drawings of objects that he 
could not identify and had no difficulty discriminating 
between complex nonrepresentational patterns, differ¬ 
ing from each other only subtly. He occasionally failed in 
discriminating because he included imperfections in the 
paper or in the printer's ink. He could never group draw¬ 
ings of objects by class unless he could first name the 
subject. Reading, both aloud and for comprehension, was 
limited greatly. He could read, hesitantly, most printed 
letters but often misread 'K' as 'R', and 'L' as 'T' and vice 
versa. He was able to read words slowly by spelling them 
aloud. 

a Hillary R. Rodman, Luiz Pesson, & Leslie G. Ungerleider. 

Excerpted from Rubens and Benson, 1971. 
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FIGURE 10.31 Effortful thought 
in word finding. These figures show both 
the location of the increased activity in 
the Tip of the tongue' state, and their time 
course (using event-related potentials). 
Notice that the classical executive regions 
are again active. While these regions are 
not normally viewed as contributing to 
conscious experience, the fact that tip- 
of-the-tongue states can be reported in a 
verifiable manner suggests that they are at 
least 'fringe' conscious. Source: Maril et al., 
2001 . 


feeling and 'feelings of knowing'. It is easy to induce 
a tip-of-the-tongue state. All we need to do is provide 
people with definitions of fairly rare but known words, 
and ask them if they feel they almost have the answer, 
but not quite. Effective questions might include 'what 
is the name of a vegetarian dinosaur'? or 'what are two 
words for the technology for making artificial limbs'? 

Such subjectively vague but reportable events 
have been found to guide intuitive problem solving, 
including verbal and pictorial problems (Bowers et al., 


1990), promote persistence in memory search during 
tip-of-the-tongue states (Brown and MacNeill, 1966), 
guide memory retrieval and persistence (Metcalfe, 
1986), and influence judgment tasks and decision 
making (Yzerbyt et al., 1998). Only a few brain studies 
are available, but Maril et al. (2001) have shown that the 
tip-of-the-tongue state, which can be easily induced, 
showed high activity in prefontal regions, the same cor¬ 
tical areas that are associated with persistence in prob¬ 
lem solving (Duncan and Owen, 2000) (Figure 10.31). 
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FIGURE 10.32 Sudden insight in problem solving. Alpha and 
gamma density in decomposed EEG at the moment of insight in a 
word association task. Notice that just before a correct response, 
alpha density declines just as gamma density increases. Gamma 
activity is thought to be due to active and synchronized process¬ 
ing in a network of related regions needed to solve the problem. In 
(c), immediately before the solution (marked with minus values in 
seconds), there is an EEG shift to the left hemisphere. Source : Jung- 
Beeman et al., 2004. 


There is good evidence that spontaneous thought 
is problem oriented, even if the goal of the mental 
process is not explicit (Greenwald et al., 2002). The con¬ 
tents of spontaneous thoughts are often described as 
one's 'current concerns'. Social psychology experiments 
have found that one can experimentally prime some 
current concern, such as relationship arguments, by 
showing videos that 'play out' those concerns. Thus, 
college students who rated their relationships with their 
parents as conflictual reported more common thoughts 
related to conflict when they were provoked by a video. 

However, similar problem solving patterns are 
found to be very common when careful studies have 


been performed on the routine 'stream of conscious¬ 
ness' (Singer, 1994). 'Jumps' in the flow of thought are 
apparently routine, and they can be elicited experi¬ 
mentally in tip-of-the-tongue experiments (see Figure 
10.31) and the well-known Remote Associates Test 
(Kihlstrom, 1996). Sudden insight can also be elicited 
experimentally, as shown in Figure 10.32. 


6*0 SUMMARY AND CONCLUSIONS 


Working memory is the domain of problem solving. 
Completely explicit problem solving is probably rare in 
the natural world. However, explicit puzzles like the 
'tower problems' are quite sensitive to frontal lobe 
damage and to other impairments of problem solving 
capacities, like drowsiness, drug effects, boredom, cog¬ 
nitive overload, or distraction. Thus, explicit problem 
solving tasks are useful indices of brain functioning. 
In addition, tasks like the Wisconsin Card Sorting Test 
can be used to set up 'fixed' expectations about a puz¬ 
zle, to allow testing of cognitive flexibility in the face of 
unexpected changes in tasks. Subjects with prefrontal 
impairments are again vulnerable to such task shifting. 

Because of capacity limits of working memory, 
attention, conscious processes, and voluntary control, 
a major strategy in problem solving is to use chunk¬ 
ing or other long-term memory components to shift 
routine aspects of problem solving to the large-capacity 
memory systems. Chess experts, for example, know 
a great many predictable chess positions from mem¬ 
ory, freeing their working memory capacity to deal 
with novel and unpredictable aspects of a chess game. 
Long-term semantic memory is still rather mysterious 
in its details, but is known to use temporal and pre¬ 
frontal regions, as well as the episodic learning capaci¬ 
ties of the medial temporal lobe. 

While implicit thinking is efficient, it is also vulner¬ 
able to rigidity and lack of flexible control. An optimal 
problem solving strategy mixes explicit and implicit 
approaches. Fringe-conscious judgments are com¬ 
monly encountered in tasks like 'feelings of knowing' 
and 'tip of the tongue'. These tasks may give us met- 
acognitive knowledge of ongoing implicit problem 
solving processes. They are now believed to activate 
the classical executive regions of the prefrontal cortex. 
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7*0 DRAWINGS AND STUDY 
QUESTIONS 

1 Fill in the boxes in Figure 10.33. Notice that the 
boxes are paired. One of each pair refers to a 


brain region and the other to a major possible 
function discussed in this chapter. 

2 Do the same for Figure 10.34. What is the rela¬ 
tionship between the two figures? 

3 Give an example of 'chunking' in an expert task. 
Why is chunking necessary in the human brain? 



FIGURE 10.33 
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Mr Broca , on the occasion of this report , presented the brain of a fifty-one-year-old man who 
died in his care at Bicetre hospital , and who had lost the use of speech. . . . When the patient 
was admitted to Bicetre , at the age of 21, he had lost, for some time , the use of speech; he could 
no longer pronounce more than a single syllable , which he ordinarily repeated twice at a time; 
whenever a question was asked of him, he would always reply tan , tan, in conjunction with 
quite varied expressive gestures. For this reason , throughout the hospital , he was known only by 
the name of Tan. 


Pierre Paul Broca (1861), Loss of Speech, Chronic Softening and Partial Destruction of 
the Anterior Left Lobe of the Brain, First published in Bulletin de la Societe 
Anthropologique , 2, 235-238. Translation by Christopher D. Green 
http://psychclassics.yorkuxa/Broca/perte-e.htm 



Brain activity to a single spoken word (starting with the first panel at the top). Notice the fast 
spread of activity beyond the auditory cortex. By the third image down, there is spread forward 
to the left inferior frontal gyrus (L-IFG), as well as central sulcus, parietal cortex, and even the 
occipital pole. These MEG scans show mismatch negativity (MMN), a large wave of electro¬ 
magnetic activity in response to an unexpected stimulus. Source : Pulvermiiller et al., 2006. 
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L0 INTRODUCTION 

Language is the foremost tool of human thought and 
culture. It is also one of the major landmarks of child 
development, with no close parallel in other species. 
Before our fourth birthday we have solved the prob¬ 
lems of understanding our first phonology, our first 
basic lexicon, and syntax. New words are acquired at 
a very fast pace during those years. While our under¬ 
standing of syntax is still not fully settled, children 
acquire it with little visible difficulty In addition, 
young children know how to use their emerging lan¬ 
guage skills to accomplish important goals (Figure 
11.1). Even before they acquire language they have a 


good understanding of the world around them. These 
are achievements of brain development, enabled by 
good caregivers, helped by cognitive and emotional 
stimulation and experience, and of course by the cul¬ 
tural gift of a native language. 

To illustrate the scientific questions posed by lan¬ 
guage, it is interesting to ask someone to repeat a sen¬ 
tence he or she has just heard. You may be surprised: 
people can rarely remember a sentence verbatim after 
only a few seconds. It's not that we have poor memo¬ 
ries, but rather that we tend to retain the meaning, not 
the words of what we hear (Sachs, 1967). Most people 
can therefore give us a paraphrase of what they hear - a 
different sentence with a similar meaning - but not the 
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Levels of language processing 



FIGURE 11.1 Levels of 
language - analysis and produc¬ 
tion. A sketch of levels of language 
analysis and production. Each level 
is highly complex, but is processed 
by skilled speakers largely uncon¬ 
sciously and in seconds. Source : 
Baars, adapted from Miller, 1991. 


original sentence. As soon as speech is understood, we 
tend to forget its 'outer form'. It has served its purpose. 

Thus, within a few seconds, sound input seems to 
go through the following analyses: 

• Acoustical analysis - turning sounds into linguistic 
elements such as phonemes; phoneme coding and 
serializing, to construct syllables and morphemes 
(meaningful units) 

• Lexical identification - assigning words to the 
input, chosen from a vocabulary of tens of thou¬ 
sands of words 

• Syntactic analysis - identifying nouns, verbs, and 
other grammatical categories, and constructing a 
syntactic frame 


• Semantics - building the semantic network of the 
lexical and syntactic structure 

• Discourse and conversational reference - how does 
the identified meaning relate to previous concepts 
in the conversation or discourse? 

• Pragmatic and social inferences - what is the 
speaker's goal, and what does it mean for my 
goals? 

2.0 THE NATURE OF LANGUAGE 

As Barrett et al. (2003) have put it, monkeys think 

'what now'?, but apes think 'what if'? That is, primates 
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with larger frontal cortices can think about imagined 
events, not in the here-and-now, but in the there- 
and-then as well. It seems like a small difference, 
but language and a host of language-based cultural 
developments have turned it into moon landings and 


toothbrushes, and perhaps some developments that 
are not quite as pleasant. The evolutionary history of 
the human species is, in many ways, the history of the 
deployment of language in pursuit of personal and 
cultural ends. 


BOX 11.1 Ambiguities exist at all levels of language 


Figure 11.1 gives the impression that there is a point-to- 
point relationship between sound input and phonetic 
analysis or between words and their meanings. This is an 
impression expert speakers tend to have, and it is wrong. 
Language is rife with ambiguities at every level of anal¬ 
ysis, both in input and output. That is to say, there are 
choice-points in processing (see Chapter 10). For example, 
most common words have more than one meaning, so 
that the mapping between the lexicon and the conceptual 
representation at the top of Figure 11.1 always involves 
alternatives. There are also frequent ambiguities in acous¬ 
tical analysis, and famous syntactic ambiguities, like the 
tree diagram of a surface structure ambiguity in Figure 
11.2 (Chomsky, 1957). 

A recent estimate of lexical ambiguity suggests an 
average of two high-frequency interpretations per word 
(Miller, 1991). More common words tend to have more 
meanings, so that the word 'set', for example, has more 
than two dozen senses according to the Oxford English 
Dictionary (Simpson and Weiner, 1989). 

Choice-points also exist in the production of speech 
and language. We tend to become conscious of them in 
creative writing, when we perceive a choice between two 
words with slightly different meanings. Synonyms and 
paraphrases create choice-points in language production, 
while ambiguities at different levels of analysis create 
choice-points in language input. 

Choice-points in the flow of processing require reso¬ 
lution, often using different levels of analysis. They turn 
a simple flow from one linguistic level to another into a 
maze, which tends to lead to a 'combinatorial explosion'. 
That is, for n binary choice points, the number of possi¬ 
ble paths through the maze rises exponentially as 2 n . This 



ADJ N 


They are flying planes 



They are flying planes 


number rises so quickly that computer-based speech and 
language interpretation has proven to be very difficult. 

The general solution is called 'top down' or 'expecta¬ 
tion-driven' processing. That is, when we encounter an 
ambiguous word like 'set' in 'Please set the table', we can 
tell from the semantic context that dishes are involved. 
In 'Please, set an example', something very different is 
intended. In these cases, semantic knowledge is needed 
to resolve the ambiguity. Thus, higher levels of analysis 
are needed to resolve choice-points at lower levels. The 
'hierarchy' of language, like the processing hierarchy 
of vision (Chapter 3) requires flow of information in all 
directions. The flow of information might look something 
like Figure 11.3. 

In general, as Mesalum (1990) has written: 

Cognitive problems are not resolved by a sequen¬ 
tial and hierarchical progression toward prede¬ 
termined goals but instead by a simultaneous and 
interactive consideration of multiple possibilities 
and constraints until a satisfactory fit is achieved. 
The resultant texture of mental activity is char¬ 
acterized by almost infinite richness and flexibil¬ 
ity. According to this model, complex behavior is 
mapped at the level of multifocal neural systems 
rather than specific anatomical sites, giving rise to 
brain-behavior relationships that are both localized 
and distributed. 

Top-down or expectation-driven processing appears to be a 
universal property of the cognitive brain. 


FIGURE 11.2 A syntactic ambiguity. There are two ways to 
understand the sentence. They are flying planes', either as The 
pilots are flying planes' or The planes are flying'. The assignment 
of the subject (they) is ambiguous, and as a result, the underlying 
structure is as well. Pronouns like 'they' are a rich source of ambi¬ 
guity in language, since they take the referent for granted. Skilled 
speakers are rarely conscious of such ambiguities, but newcomers 
to any language community tend to misunderstand them. Syntactic 
ambiguities are just one kind of choice point in language analysis. 


(Continued) 
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2.1 Biological aspects 

Virtually all humans learn to speak in the first several 
years of life, and no other species does. There are inter¬ 
esting exceptions, like the chimps and gorillas, who are 
able to acquire sign language by observational learning 
and training early in life. But, as a species, other liv¬ 
ing primates do not acquire language, while humans 
do. This is not to minimize the special capacities of 
bonobos and other primates, nor of other species, like 
the African grey parrots Alex and Arthur (Pepperberg, 
2002); it is simply that language appears to be a human 
capacity for which we are biologically and culturally 
prepared. That is why we are good at it. 


Children acquire language in predictable stages, and 
even develop their own 'creoles', true rule-governed 
languages, when they grow up in mixed-language 
communities without a single dominant language (see 
Box 11.2). After childhood it becomes more difficult to 
acquire the skills of a native speaker. Spoken language 
is a complex biological overlay over pre-existing vocal 
and auditory physiology. Language production tends 
to be lateralized to the dominant hemisphere (usually 
on the left side), though early brain damage can cause 
a shift to the non-dominant side (Chapter 1). The gene 
FOXP2 apparently needs to be expressed accurately for 
normal human speech to develop, although that gene 
















































2.0 THE NATURE OF LANGUAGE 


375 


is widely found throughout vertebrate species (Vargha- 
Khadem et ah, 2005). 

Repeating a sentence requires specific brain 
regions, assuming that the brain has developed nor¬ 
mally. The best-known examples are Broca's and 
Wernicke's areas (Chapter 1), but speech and language 
recruit much wider cortical and subcortical activity. 
'Language cortex' has only become specialized for 
speech within hominid evolution, less than 3 million 
years out of the 200 million years of mammalian evo¬ 
lution. Prior to hominids, Broca's area may have been 
involved with vocal tract control for other purposes: it 
is located, after all, immediately adjacent to the mouth 
and tongue control areas of the motor homunculus. 
Many ingredients of spoken language, like hearing 
and vocal control, must have emerged very early. But 
there is speculation that the full panoply of language 
abilities may have become available in a much shorter 
time span than 3 million years, perhaps as short as 
30-100 000 years. Certainly, the working vocabulary 
of living languages seems to have expanded consid¬ 
erably in recent history, although syntax may have 
become simpler. 

All these points suggest that speech and language 
involve biological preparedness, in much the way 
that manual dexterity, vision, brain development, and 
social relationships do. Like other genetic influences, 
this one is enormously interactive. Which language 
we acquire in childhood, how well we learn it, what 
vocabulary (and consequently conceptual system) we 
acquire, and many other variables depend upon our 
experiences. Nevertheless, the biological substrate of 
language is an important source of insight, as we will 
see. It does not reduce humans to some simpler liv¬ 
ing species, but rather should be taken to emphasize 
the enormous complexity and adaptive significance of 
language. Our cultural and personal accomplishments 
are heavily dependent upon the biological gifts of our 
species. 

Chapter 1 touched on the discovery of the brain 
areas for speech production and perception in the 
19th century, by Pierre-Paul Broca and Carl Wernicke, 
respectively. However, their findings with a small 
number of brain-damaged patients touched off more 
than a century of debate on the question of localiza¬ 
tion of language in the brain. That debate continues 
today, often involving neural net models of language 
functions, for example, and sophisticated brain record¬ 
ing methods (Chapters 3 and 4). 

Figure 11.4 shows the classical understanding of 
Broca's area for speech production, and Wernicke's 
area for speech perception and comprehension. In 


addition, physicians in the 19th century discov¬ 
ered a number of other aphasias (language deficits). 
The best known of these is conduction aphasia, associ¬ 
ated with a deficit of the arcuate fasciculus, the 'arched 
little bundle' running between Broca's and Wernicke's 
areas (Catani and Ffytche, 2005). One of the most 
important developments today is the ability to study 
the large white matter tracts that run between corti¬ 
cal areas, and which fill by far the greatest amount of 
space in the hemispheres. Tractography (Chapter 4) 
and other methods for studying brain connectivity in 
the living brain should make it possible to learn much 
more about both cortical specializations and their con¬ 
nections in the next several years. 

Wernicke's area abuts the auditory cortex in the 
Sylvian fissure and superior temporal gyrus (STG in 
Figure 11.4). Broca's area is immediately adjacent to 
the mouth and vocal tract regions of the motor cortex. 
Things have gotten a lot more complex in our under¬ 
standing of these regions, but it is useful to realize that 
the location of these areas in those particular neigh¬ 
borhoods makes functional sense. 

In 1861, when Broca discovered patients with dam¬ 
age to the left inferior frontal gyrus (L-IFG), which 
is now called Broca's area, a great debate broke out 
among neurologists who found it hard to find patients 
with exactly the same damage and the same symp¬ 
toms. That debate lasted well into the next century. We 
might expect the debate to be resolved with the use of 
sophisticated brain imaging methods, or with direct 
cranial stimulation studies of hundreds of patients in 
open brain surgery (Penfield and Roberts, 1959; for a 
review, see Ojemann, 2003). Surprisingly, it has not. 
Debates about localization of function continue today. 

That is not to say that nothing has been learned in 
more than a century of research. Most scientists believe 
that Broca's area is necessary for normal speech produc¬ 
tion. But a lot more is going on in this part of the brain 
than previously suspected, and there are cases of dam¬ 
age to Broca's area that do not show the classic inability 
to speak. As improving neuroimaging methods are able 
to examine smaller and smaller regions of cortex, new 
subdivisions of the left inferior frontal gyrus (LIFG) are 
constantly being proposed. 

The brain doesn't file its memory stores in neat, 
separate locations. However, there may be a general 
tendency for long-term functions to activate related 
sensory, motor, motivational, and language regions. 
The cortex has numerous very local connections 
within single vertical columns and sets of columns; it 
has slightly more remote connections between neigh¬ 
boring regions of larger size; and it has a vast highway 
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FIGURE 11.4 Classical language 
regions of cortex. The traditional location 
of Broca's and Wernicke's areas of the left 
hemisphere, based on neurological patients 
showing production and comprehension 
deficits. The right hemisphere has receptive 
language functions that are not shown here. 
(STG: superior temporal gyrus; CS: central 
sulcus.) Source : Standring, 2005. 


system of remote connections coursing from one end 
of the brain to the other, from the posterior to anterior 
poles, from right to left hemispheres, and flowing 
upward and downward into the sensory and motor 
apparatus of the cranial nerves and the spinal cord 
(see Chapter 5). This kind of highly interconnected 
system looks more like the world-wide web than like 
the organizational chart of a college campus. 

Before we leap to the conclusion that language and 
thought are represented in easily identifiable locations 
in the brain, it is useful to look at a large-scale sum¬ 
mary of the literature by Vigneau et al. (2006). Figure 
11.5 shows the results of a major meta-analysis of 
more than 125 brain imaging experiments, resulting in 
more than 700 identified regions of peak activity in the 
left hemisphere. The different colors represent phonol¬ 
ogy, semantics (concept-related activity), and sentence 
or text processes. The degree of overlap is remarkable. 
This is not the kind of pattern we encounter in study¬ 
ing sensory regions like vision, where clearly defined 
visual maps like VI can be shown to map topographi¬ 
cally on to the retina. 

These facts have a strong theoretical interpretation. 
They suggest that much of the brain works by way 
of distributed networks of language functions, like the 
world-wide web. An Internet chat group may work 
even if the participants come from many different parts 
of the world. The idea of distributed brain functioning 
seems to support the connectionist view of the brain, 
as discussed in Chapter 3 (see also Neural Darwinism, 
Chapter 3). 

By performing a statistical cluster analysis of hun¬ 
dreds of data sets, Vigneau et al. were able to suggest 
more specific loci, interacting in classical working 
memory loops (Figure 11.6). The analysis suggests 
three separable working memory loops, for phono¬ 
logical, semantic, and sentence processing. But now 
we are left with a puzzle that has occupied scientists 
at least since Pierre-Paul Broca: which figure tells us 
what's really happening? It is possible that the statisti¬ 
cal analysis shows the 'true' nature of the regions in 


Figure 11.6. That would be a sensible interpretation if 
there were random jitter in the data for methodologi¬ 
cal or biological reasons. But each data point in Figure 
11.5 represents the best efforts of each laboratory to 
ensure that there is no such random jitter. Thus we 
have a choice between accepting either a more locali- 
zationist account suggested by the cluster analysis, or 
the distributionist account suggested by Figure 11.6. 
It is not obvious which one is correct, and the debate 
continues. 

Nevertheless, there is good agreement on some 
basic points. For example, both figures show constant 
interactive looping between the more sensory-related 
(posterior) and motor-related (anterior) parts of cortex. 
While the left hemisphere is usually studied because 
of the well-established left-side bias in most people, 
it is important to understand that language functions 
are not confined to the left hemisphere. There is good 
evidence for speech and language input processing on 
both sides - speech perception and comprehension. 
But for reasons that are not well understood, language 
output, the planning and control of speech, is weighted 
toward the left hemisphere in more than 90 percent of 
the population. As we will see later, the right hemi¬ 
sphere may even have its own way of understanding 
sophisticated communications like jokes, metaphors, 
and irony, while the left side may have a more hum¬ 
drum preference for literal language (Zaidel et al., 
2000 ). 

We will see these ideas again as we explore how 
the brain supports human language and thought. 
Various scientists have suggested somewhat different 
interpretations of the very large empirical literature. 
We have carefully collected evidence from studies 
of brain damage, single-neuron recording, positron 
emission tomography (PET), functional magnetic 
resonance imaging (fMRI), magnetoencephalogra¬ 
phy (MEG), electroencephalography (EEG), evoked 
potentials, and the like. Not all the evidence is easily 
reconciled, but it is useful to take a broad organizing 
perspective. 
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FIGURE 11.5 Widely distributed language networks may be 
compatible with regional specialization. A summary of more than 100 
brain imaging experiments, reflecting some 730 activity peaks using 
fMRI and PET (Vigneau et al., 2006). The blue points show peaks for 
phonology - the sounds of speech. Red regions show peak regions 
for semantics, the meanings of words and phrases, and the green dots 
show the effects of sentences and text. The overlap and wide scatter 
of the three functions is striking. Source : Vigneau et al ., 2006. 


For example, Hagoort (2005) suggests that we 
should take a larger view of what is classically called 
Broca's area. He proposes a division of labor as shown 
in Figure 11.7. Broca's area has now expanded for¬ 
ward to include the gray oval area, which is proposed 
to serve the function of unifying speech sounds, mean¬ 
ing, and syntactic relationships. This area is intended 
to include both memory areas of the temporal lobe, 
including the MTL (medial temporal lobe, which is 
hidden in the left viewpoint). Finally, an executive 
region includes the dorsolateral prefrontal cortex 
(DL-PFC). 

As we know from the dramatic case of young chil¬ 
dren with left hemisphere surgeries, the left-side bias 
for language can switch to the other side if there is 
severe damage to the speaking hemisphere in early 
childhood (see Chapter 2). Lateralization of func¬ 
tion is also found in some other species, but the rea¬ 
sons for it are still quite mysterious. For this topic, it 
is only important to understand that the wide distri¬ 
bution of language activity points is not limited to the 
left hemisphere. It expands like some vast metropo¬ 
lis into the non-speaking hemisphere. Anatomically, 
the great bridge between the hemispheres, the cor¬ 
pus callosum, fans out on both sides in a point-to- 
point fashion, so that neurons in the frontal cortex 
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FIGURE 11.6 Working memory loops for phonology, seman¬ 
tics, and sentences. The wide scatter of peak activations in Figure 
11.5 can be simplified by a cluster analysis, resulting in plausible 
centers of speech-related activity in the left hemisphere. However, 
such an analysis must then explain why the wide distribution of 
phonology, semantics, and sentence processing was found in the 
first place. The answers are not obvious at this time. (STSp: superior 
temporal sulcus, posterior; AG: angular gyrus; SMG: supramarginal 
gyrus; F2p: middle frontal gyrus, posterior; PrF3op: operculum 
of inferior frontal gyrus; F3td: inferior frontal gyrus). The word 
"operculum' refers to a flap of cortical tissue that covers the hidden 
regions of the insula and Sylvian fissure. Frontal gyri are numbered 
downward (superior = FI, middle = F2, inferior = F3). Source : 
Vigneau et al, 2006. 


on one side sprout axons that spread across to the 
frontal cortex on the other side (see Chapter 5). For 
many functions, the two hemispheres are thoroughly 
integrated. 

It is still true that damage to Broca's area will tend 
to impair speech output, while damage to Wernicke's 
area and neighboring regions tends to degrade speech 
comprehension (Figure 11.8). Historically, Broca and 
Wernicke were not wrong on the evidence - they just 
didn't have the complete story. 

In Figure 11.11, we present a model for audi¬ 
tory language in the cortex proposed by Hickok and 
Poeppel (2007). Notice that the upper regions of the 
temporal lobe, toward the back of the Sylvian fissure, 
contain auditory regions that are believed to repre¬ 
sent phonemes or possibly the 'syllabary' mentioned 
above. These are the sound-based representations of 
speech (Figure 11.11). This is a plausible model, but as 
mentioned, we do not yet have the spatial resolution 
to know whether there is a more microscopic mosaic 
of feature-sensitive neurons, such as exist in visual 
cortex. 








378 


11. LANGUAGE 


BOX 11.2 Creole languages are produced by 
children in multilingual communities 


Creoles are languages with simple syntactic structures 
that develop when speakers of several different lan¬ 
guages are forced to communicate with one another. 
This can occur through immigration and invasion or 
when individuals are brought together from differ¬ 
ent cultures to work. The adults in such situations 
develop a crude pidgin, an impoverished communi¬ 
cation system in which a limited number of nouns, 
verbs, and modifiers are combined with extensive ges¬ 
turing. Children of the pidgin speakers learn their par¬ 
ents' language, but not the pidgin; instead, in addition 
to using their parents' language, they also create a cre¬ 
ole for communication with one another that is based 
on the pidgin. All creoles share a set of common fea¬ 
tures, which are surprisingly similar to the features of 
fully developed languages. Thus, the children develop 
a much more full-fledged language than their parents 
in these communities. 

1 The grammar is based on word order; in contrast, 
most human languages rely largely on inflection 
(agreement and derivation - see earlier discussion). 

2 There are seven parts of speech: nouns, pronouns, 
adjectives, verbs, adverbs, articles, and conjunc¬ 
tions; many human languages omit one or more of 
these categories or add new ones. 

3 Nouns are distinguished as singular, plural, or 
indefinite in number; many languages lack the lat¬ 
ter category. 

4 There are three particles used as auxiliary verbs 
to indicate whether an action is successful, unsuc¬ 
cessful, or repeated; many languages lack these 
distinctions. 

5 There is a single verb conjugation system; most 
languages have a general rule for most verbs, but 
special-case rules for most of the commonly used 
(irregular) verbs. 

6 Questions are based on intonation rather than word 
order; many languages use both. 

Bicker ton points out that, in many cases, these uni¬ 
versal features of creoles are not present in the lan¬ 
guages spoken by the parents of the children who 
created the creole. Therefore, creoles are very likely to 
be related to innately specified features of language. 

However, even creoles do not illustrate such features 
directly. If all the features of creoles just listed were 
innate, it would be impossible to explain how any lan¬ 
guages have any other features. Innate properties of 
language must be more abstract than the features found 
in creoles and must allow all the different forms of lan¬ 
guage that are currently found to develop. Instead of 
thinking of creoles as demonstrating universal features 
of language directly, we might better think of them as 
showing us what features of language develop most eas¬ 
ily. It is possible that these features characterized human 
languages at an early stage of their development. 

From Squire et al. (2003). 


2.2 Language origins 

People have wondered about the origins of language 
for thousands of years. If children grew up with¬ 
out language, would they invent their own? In fact, 
nature provides experiments a little bit like that. In 
some isolated communities, like small islands, adult 
speakers of different languages may settle in the 
same place. Fluent adults rarely acquire the 'accent' 
(phonology) or grammar of languages they learn 



FIGURE 11.7 L-IFG: an expanded concept of Broca's area. 
The left inferior frontal gyrus (L-IFG) is a more accurate label for the 
broad area involved in speech planning and production. Hagoort 
(2005) also suggests that L-IFG is a 'convergence zone' for speech, 
i.e. it is a place where the different features of spoken language are 
unified into an integrated plan before being sent to the motor map. 
Source : Hagoort, 2005. 



FIGURE 11.8 Wernicke-Geschwind model. The best-known 
neurologically based model of the language regions is due to 
Geschwind (1979). While the Wernicke-Geschwind model continues 
to be widely studied, the advent of neuroimaging methods has led 
to a wave of new evidence. Source : Weems and Reggia, 2006. 
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after puberty. Instead, they tend to speak a 'broken' 
version of the new language, one that is good enough 
to communicate with adults from other parts of the 
world, but generally without the fluency and expres¬ 
sive richness of their original languages. Children in 


such bicultural communities do a very interesting 
thing: they may develop a language of their own, 
now called a 'creole' because they were first studied 
in the Creole communities of the Caribbean Islands 
(Box 11.2; Bickerton, 1984,1990). 


FRONTIERS OF COGNITIVE NEUROSCIENCE 


How spoken sentences are processed in the brain 


* f J 
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FIGURE 11.9 Angela D. Friederici, PhD, Max Planck Institute 
for Human Cognitive and Brain Sciences, Leipzig, Germany. 

We communicate mostly by spoken language. Spoken 
sentences contain not only semantic and syntactic infor¬ 
mation, conveying the meaning of words and their gram¬ 
matical relationships (who is doing what to whom), but, 
moreover, prosodic information - the melody and rhythm 
of language. Prosody can provide information signal¬ 
ling the boundaries of syntactic phrases, and it can also 
tell us about the emotional tone modulating meanings. 
Neuroimaging indicates that semantic and syntactic pro¬ 
cesses are supported by neural networks in the left hemi¬ 
sphere, comprising inferior frontal regions (Broca's area 
and more anterior parts) and superior and middle tem¬ 
poral regions (largely Wernicke's area) (Friederici, 2002; 
Hickok & Poeppel, 2007). The functional network for 
syntax includes Broca's area (Stromswold et al., 1996) and 
the most posterior portion of the superior temporal gyrus 
and sulcus (Bornkessel et al., 2004); semantics recruits 
brain regions anterior to Broca's area in the inferior fron¬ 
tal gyrus (Thompson-Schill et al., 1997) and portions 
of the middle and superior temporal gyrus (Demonet 
et al., 1994). For syntactically complex sentences, we see 


activation in the network that connects Broca's area and 
the posterior superior temporal gyrus (Friederici et al., 
2006a). Recent work indicates that within this fronto¬ 
temporal network Broca's area serves the function of 
processing syntactic hierarchies independent of the 
sequence's meaning (Friederici et al., 2006b; Makuuchi 
et al., 2009). The posterior temporal region comes into 
play when processing syntactically complex meaningful 
sentences (Bornkessel et al., 2004; Friederici et al., 2009). 
The last finding suggests that the posterior temporal cor¬ 
tex helps to integrate meaning and syntax in sentence 
comprehension. 

Prosodic information - the melody of language - 
is processed in the right hemisphere, especially the 
superior temporal and inferior frontal regions (Meyer 
et al., 2002). Event-related brain potentials (ERPs) 
indicate that intonational phrase boundaries can guide 
or mislead our ability to parse sentence syntax when 
listening to sentences (Steinhauer et al., 1999). For 
example, the two sentences: (a) Since John always jogs two 
miles this seems like a short distance to him and (b) Since 
John always jogs two miles seems like a short distance to him 
are clearly syntactically different. If you read them care¬ 
fully you'll see that they are both meaningful, nor¬ 
mal sentences - they just have a different intonational 
boundary. In sentence (a) we can express the end of a 
phrase after 'miles'; in (b) we change our intonation 
after 'jogs'. An intonational break after 'jogs' in sentence 
(a) would mislead our ability to parse the syntax of the 
sentence. 

This suggests a strong interaction between the left 
hemisphere's syntactic processing, and the right hemi¬ 
sphere's prosodic processing. That interhemispheric inter¬ 
play is guaranteed by fiber bundles connecting the two 
hemispheres; that is, the corpus callosum. ERP data from 
patients with lesions in the posterior third of the cor¬ 
pus callosum indicate that this brain structure is crucial 
for the interplay of syntactic and prosodic information 
(Friederici et al., 2007). It allows the two information types 
to interact early on, to help speed our understanding 
of spoken sentences (Eckstein et al., 2006). 
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FIGURE 11.10 A schematic view of the language regions in 
the left hemisphere, with major relevant gyri and sulci. Numbers 
indicate language-related Brodmann areas (BA), defined on the 
basis of their microscopic cellular characteristics (Brodmann, 1909). 
The classical language areas are Broca's area (BA 44 and 45) and 
Wernicke's area (BA 22). The words 'superior' or 'inferior' refer to 
the position of a gyrus within a lobe (e.g. superior temporal gyrus) 
or within a Brodmann area (e.g. the superior BA 44). The terms 
'anterior' or 'posterior' indicate the position within a gyrus (e.g. 
the posterior superior temporal gyrus). Source: Friederici. 
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(a) Via higher-order frontal networks 




FIGURE 11.11 Hickok-Poeppel model 
of auditory language. Upper panel shows a 
schematic for the model of auditory language 
processing proposed by Hickok and Poeppel 
(2004). Lower panel shows brain regions pro¬ 
posed to reflect stages of the model. Note that 
early speech perceptual systems for mapping the 
acoustic-phonetic information in sounds onto 
meaning are proposed to be mediated bilaterally 
in left and right hemispheres while later proc¬ 
esses are proposed to be mediated by left hemi¬ 
sphere regions. Source : Adapted with permission 
from Hickok and Poeppel, 2007. 
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2.3 Speech versus language 

Scholars often make a distinction between speech and 
language, in part because of the striking plasticity of 
the human brain in learning different input and out¬ 
put modalities. Right now you are using your eyes 
and perhaps your hands, to take notes. You have 
learned to read and write or type with quite remark¬ 
able facility. You could also learn sign language, 
Braille, and paralinguistic symbol systems like math¬ 
ematics, logic, and computer programming. Thus, the 
purely vocal and auditory nature of speech seems to 
be unnecessary. 

However, a hard-and-fast distinction between 
speech and language is not justified. Human brains 
are pre-adapted to acquire auditory and vocal speech. 
The great majority of children learn spoken language 
in the first years of life, but reading and writing come 
later and have a lower rate of success. The fact that we 
can exercise remarkable linguistic flexibility, given the 
need and the opportunity, does not falsify the biologi¬ 
cal primacy of spoken language. 


3.0 THE SOUNDS OF SPOKEN 
LANGUAGE 

Chapter 7 discussed the auditory nature of speech, 
consisting of fast-expanding three-dimensional bub¬ 
bles of high and low air pressure waves, which set 
the delicate membranes of the inner ear into vibrating 
motion. We can also look at speech from a vocal output 
point of view. 

The human vocal tract is basically a tube, with two 
flexible flaps just above the lungs and its diaphragm 
muscles, which together create air pressure when we 
breathe out (Figure 11.12). The vocal tract is there¬ 
fore much like a reed horn like a saxophone, with the 
reed vibrating at the top of the tube. The vocal flaps 
can vibrate faster or slower, producing higher- or 
lower-pitched sounds. We produce consonants by 
closing the entire vocal tube at one of several places. 
In English we use the lips at the very front (/b//p/ 
/m/), the tongue against the palate in the back or 
middle (/g//k//ng//r/), and the teeth in the front 
(/th//the//s//z//v//w/). However, stopping the 
flow of air completely produces no sound at all, so 
that it is the fast transitions between closing and open¬ 
ing of the air flow that produces the so-called stop 
consonants (/b//p//t//d//k/g/). Other languages 


shape the air flow in somewhat different ways, but the 
physical principles of sound production are the same. 

Vocal vibrations echo through the head and body, 
triggering vibrations in all the cavities in our head and 
torso, so that by holding one's nose, for example, one 
can change the quality of the voice. Those vibrating 
air-filled body cavities also allow us to tell the difference 
between individual voices. Children's voices sound 
higher than adults' simply because they have smaller 
vibrating cavities. 

Just as closing the vocal tube creates consonants, 
shaping the vocal cavity with the mouth open pro¬ 
duces different vowels. Singing is closely related to 
speaking: it is just stretching the length of vowels and 
tuning them to a specific pitch using the vocal cords. 
But even ordinary speech has a kind of melodic phras¬ 
ing called intonation contours. In English, a question 
intonation tends to raise the pitch of the last few syl¬ 
lables of a spoken phrase. A great range of emotional 
qualities is conveyed by the intonational melodies of 
speech. 

Even the rhythm of music has a close analogue 
in vocal stress patterns, as used in poetry and rap 
music, for example. Thus, singing and speech into¬ 
nation, rhythm, and vocal gestures can be viewed as 
using the same voice instrument in somewhat differ¬ 
ent ways. 

In sum, humans can shape the vocal tract in a great 
variety of ways, a rare (but not unique) adaptation 
among animal species. 

Until the invention of typewriting, speaking was the 
fastest moving and most precise skill that most people 
ever acquired. As Chapter 7 points out, the difference 
between the syllable /pa/ and /ba/ involves only a few 
tens of milliseconds between the onset of vocal vibra¬ 
tions and the opening of the lips. In the case of /pa/ the 
lips open slightly before voicing begins, while for /ba/, 
voice onset starts just before the lips open up. Because 
we are skilled readers and writers, we tend to believe 
intuitively that language is made up of sequences of let¬ 
ters, but that is a misunderstanding. Spoken language 
involves a series of articulatory gestures, which shape 
the moment-to-moment frequency distribution of air 
vibrations emanating from our vocal tract. While we 
can see isolated visual letters (as on this page), there is 
no such thing as an isolated vocal consonant. We can¬ 
not pronounce /k/ without making a following sound, 
even if it is just a long /hhhh/, simply because the tube 
needs both to close and open to produce any sound. For 
that reason, the physics of sound suggests that simple 
syllables (like /ba/ and /pa/) and isolated vowels may 
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FIGURE 11.12 The human vocal tract makes use of pre-existing mechanisms of breath control, mouth, tongue, glottis, and larynx inher¬ 
ited from ancestral species. The vocal tract is a tube with a source of tuned vibrations in the vocal cords, two flaps of tissue in the larynx. The 
quality of vocal sounds results from vibratory resonance with the movable tissues and air pockets throughout the head and torso. While con¬ 
sonants involve restriction of closing of the air tube, vowels are mostly shaped by moving the tongue and lips to shape the oral cavity. This has 
the effect of changing the resonant frequencies or formants of the vocal tract. Thus vowels and consonant-vowel pairs are the minimal physical 
units of speech production (if one considers hissing sounds like /h//th//tha//s/ and so on to be vowel-like). Source : Standring, 2005. 


be the simplest elements of speech. For that reason some 
theorists suggest that humans make use of a syllabary - 
not just an alphabet of phonemes, but a larger set of 
speech gestures of vowels and consonant-vowel combi¬ 
nations (Levelt and Wheeldon, 1994). A syllable alpha¬ 
bet would have to be larger than a set of phonemes, but 
it would reflect the way we produce speech sounds. 

It follows from these points that the phonemes of 
human language are abstract percepts, much as vis¬ 
ual objects are. Our eyes don't see trees - rather, they 
receive retinal projections of light and dark that are 


interpreted by the cortex as three-dimensional trees at 
a certain distance from the eyes. Similarly, phonemes 
are abstract categories of sounds, which are acousti¬ 
cally quite different depending upon their neighbor¬ 
ing speech gestures. When speakers say /ba/ versus 
/bee/, the acoustical information for the lip-opening 
that produces the consonant /b/ is quite different 
in the two cases. In the case of /bee/ there is a fast -rising 
transition to signal the consonant, while for /ba/ 
there is a fast -dropping transition (see Chapter 7). Yet 
we perceive the consonant /b/ to be the same in both 
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FIGURE 11.13 Auditory cortex shows local activity to differ¬ 
ent speech-like stimuli. Recent research shows finer parcellation of 
the auditory and speech perception regions. With the development 
of more refined imaging tools, it is possible that localized arrays 
of speech feature-sensitive neurons will be identified, much as has 
happened in the visual cortex. Source: Langers et al., 2003. 


cases. This is typical for human languages, which tend 
to have about forty abstract phonemes, which stand for 
the much larger set of physical sounds that actually 
reach our ears. Abstract phoneme perception appears 
to be species-specific. 

Are phonemes reflected in neat fields of feature- 
sensitive neurons in the cortex? We do not know the 
answer. In the case of vision, feature-sensitive cells that 
respond to color, line orientation, and the like, were first 
discovered in the macaque monkey, using single-cell 
recording. Such experiments are not done on humans, 
and of course macaques are not biologically adapted 
for speech perception. For that reason the answers still 
await better brain imaging methods (but see Figures 
11.13 and 11.14 for recent studies investigating brain 
processes for speech and speech like sounds and ges¬ 
tures). Reports of high-resolution fMRI studies of audi¬ 
tory cortex in the macaque were published even as this 
book was going to press (Petkov et al., 2006). In the 
next several years we may find out if there are feature- 
sensitive fields of auditory neurons in the cortex. 
Because the brain tends to do things similarly for simi¬ 
lar functions in the cortex, it seems like a reasonable 
prediction. But direct evidence is simply not available. 



FIGURE 11.14 Extending the tongue without making a sound. 
Even without making a sound, horizontally extending the tongue 
shows marked activity in the 'language' regions of both hemispheres. 
As this shows, the tongue and mouth are very well represented in 
motor cortex, unlike, for example, regions of the back. Motor cortex 
over-represents more important regions of motor control, and under¬ 
represents less important ones. Source : Dogil et al ., 2002. 


4.0 PLANNING AND PRODUCING 
SPEECH 

Output from the language areas of cortex is at least as 
complex and adaptive as speech perception and com¬ 
prehension. Figure 11.15 shows pathways between 
speech perception and production. Figure 11.16 shows 
a current model of the output flow, beginning with the 
conceptual level. Grammatical encoding of sentences 
is believed to involve lemmas, much like formulas 
for translating semantics into the morphemes and 
phonemes that govern vocal movements. It is impor¬ 
tant to understand again that these linguistic units 
are abstract, in the sense that any given phoneme, for 
example, can be articulated in more than one way, 
depending upon the neighboring phonemes. The 
vocal tract is a physical system, and it takes considera¬ 
ble time for the tongue, for example, to travel between 
the teeth (for shaping the consonant /th/) and the soft 
palate at the back of the mouth to make a /g/ sound. 
As a result, there is a fair amount of 'sloppiness' in 
the acoustical signal that is produced, because of the 
smearing of speech gestures. In English, for example, 
the word Tan' will foreshadow the final /n/ even dur¬ 
ing the time when the /ae/ is being pronounced. The 
/ae/ is therefore nasalized or 'co-articulated'. Thus, 
the actual sounds of speech changes, even though 
the abstract phoneme is believed to be represented 
the same way, whether we have the /ae/ in 'act' or 
in 'tan'. If you listen carefully, you will be able to hear 
the difference. 

Notice that this situation parallels the case of vision. 
In the visual cortex, area VI mirrors the output of the 
retina. It is very detailed, with small receptive fields 
and very high optical resolution. Even though later 
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FIGURE 11.15 Pathways between speaking and hearing. The results of a tractography study of the connections between production 
and perception regions. The tractographic display (above) shows the classical arcuate fasciculus running between Broca's and Wernicke's 
areas, although an additional pathway has been reported as well. The lower two figures show mathematical connection weights for the left 
and right hemispheres, respectively. (BA: Broca's area; WA: Wernicke's area; UF: uncinate fasciculus; STG: superior temporal gyrus; MTG: 
middle temporal gyrus.) Source : Parker et al, 2005. 


visual areas are also visuotopic maps, they are much 
less detailed and much more abstract in representing 
features of the visual input like color, object identity, 
and the like. The idea that the visual system becomes 
more abstract as we look upstream is therefore well 
justified by the evidence. Speech may be similar in 
that respect, although our brain evidence is far less 
complete. 

Notice also that the cortex is not the only control 
region for speaking. Actions that are initiated vol¬ 
untarily from frontal cortex also enlist regions of the 
basal ganglia and cerebellum. Traditionally, it has been 
believed that basal ganglia are involved in the auto¬ 
matic aspects of action control, such as the specific 
pronunciation of /r/ sounds compared to /l/ sounds, 
a distinction that is very difficult for native Japanese 
speakers, for example. English speakers have com¬ 
parable difficulties in controlling the French sound 
/u/ versus /oo/. These highly overpracticed speech 
sounds are not completely controlled by the cortex. In 
addition, the cerebellum is traditionally believed to be 
needed for fine motor control. 

However, recent brain imaging evidence implicates 
both the cerebellum and basal ganglia in purely cog¬ 
nitive aspects of brain activities, including working 


memory and some types of associative learning 
(Figure 11.17). 

While language areas of the cortex have been 
known since the 19th century, even the exact func¬ 
tions of Broca's and Wernicke's area are still debated 
(see Chapter 1). It isn't that Broca and Wernicke 
were wrong in their proposals about speech produc¬ 
tion and comprehension. Rather, as our observations 
become more and more precise, we are finding more 
functions for the classical language regions of the left 
hemisphere, and speech-related functions outside 
their traditional boundaries. Our understanding is 
constantly being revised. We are, in effect, exploring a 
new planet, and as our space probe comes closer and 
closer, we need to chart a much more detailed and sur¬ 
prising geography of this novel world. 


5.0 EVOLUTIONARY ASPECTS OF 
SPEAKING AND LISTENING 

Our highly specialized vocal apparatus is attuned to 
producing spoken language. It evolved from the non- 
linguistic vocal organs of ancestral species. These, in 
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FIGURE 11.16 Producing speech: from meaning to movements. A model of the production of speech. 
Compare to Figure 11.1. Source : Dogil et al, 2002. 


turn, build upon a breathing apparatus that we share 
with other land-dwelling creatures, as well as neu¬ 
romuscular control of chewing and swallowing. For 
example, tree-dwelling young gibbons in the wild sound 
very much like human children at play However, soon 
after birth human babies begin to babble, experimenting 
with syllables that are quite different from non-linguistic 
sounds (Schirmer and Kotz, 2006). 

Anatomically, vocalization involves a dual-control 
system, like breathing and emotional facial expres¬ 
sions (Chapter 2). In socially provocative situations, 
vocal sounds can be produced with minimal execu¬ 
tive control from the lateral prefrontal cortex, rang¬ 
ing from making cooing sounds to a baby, to crying 
out of sadness, shouting with anger, or groaning 
with distress. Figure 11.18 shows the dual control of 
vocalization schematically. As you can tell, it is not a 
simple system, having evolved over many millions of 
years. 

In this chapter, we are concerned with the right 
side of the diagram, the cortical control of speech 


beginning with prefrontal cortex and Broca's area 
(broadly defined), while on the input side we are look¬ 
ing at speech perception and comprehension. 

Cortical regions for speech are closely associated 
with audition (for sensory input) and with mouth and 
vocal tract representation (on the output side). The 
classical Broca's and Wernicke's areas are adjacent to 
cortical regions for vocal production and sound per¬ 
ception, respectively (Figure 11.19). Figure 11.20 shows 
the mouth and vocal region of the motor homunculus, 
immediately adjacent to Broca's area. Direct stimula¬ 
tion of the motor map results in muscular movements, 
but stimulation of premotor regions such as BA 6 
results in reports of 'urges to move' the corresponding 
part of the body. Motor map stimulation is perceived 
as externally controlled by the physician, not by the 
patient. Broca's area may be considered premotor cor¬ 
tex for speech, i.e. a cortical region for the 'intention to 
speak'. When Broca's area is stimulated in a conscious 
patient, it appears to block the intention to speak 
(Quinones-Hinojosa et al., 2003). 
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FIGURE 11.17 (a) A neural network for speech production: the supplementary motor area (1) and the cingulated motor area (2) are con¬ 

nected with the primary motor cortex (3). Subcortical activation is in the thalamus (4), the basal ganglia (not shown), the red nucleus (6), and 
the cerebellum (5). Additionally, the posterior temporal gyrus in both hemispheres is activated in speech production (7). In the brainstem, areas 
such as the nucleus hypoglossus (8) are innervated during speech production. Cortical regions work with the basal ganglia, thalamus, and cer¬ 
ebellum in speech control. Subcortical satellite regions like the basal ganglia are involved in sequential behaviors like speaking. In addition, 
the cerebellum and thalamic nuclei (not shown) play an important role. Sources: Left: Soros et al., 2006; right: Angerine, 2002. 


As more has been learned about the cortical aspects 
of speech and language, the production and perception 
regions have expanded and also become more finely 
fragmented into specialized areas. Additional functions 
have also been discovered for the classical Broca's and 
Wernicke's areas, as in the case of the 'mirror neurons' 
discussed in Chapter 14. Speech perception (but not 
production) also appears to recruit the non-speaking 
hemisphere (the right hemisphere in most people), even 
though speech planning and production is typically 
limited to the left side. 


6.0 WORDS AND MEANINGS 

English is a member of the Indo-European language 
family. This family includes Persian and a number of 
languages spoken on the Indian subcontinent. Other 
language families make quite different choices with 
respect to such basic-seeming units as words. Finnish 
and Turkic languages are called 'agglutinative', because 
they string morphemes into long utterances, like the 


unusually long English word 'antidisestablishmentari- 
anism'. Such long strings of morphemes are normal in 
agglutinative languages. Tonal languages like Chinese 
and Tibetan take the opposite approach, compounding 
sentences from typically short words, modulated by a 
rich melodic shaping of each syllable, so that a word 
like 'Chang' can mean quite different things depend¬ 
ing on its tonal contour. However, all languages have 
lexical units of some kind - utterances of one or more 
syllables that refer to meaning categories or to rela¬ 
tions between categories. 

Figure 11.21 shows a current model of how the 
brain may translate different linguistics codes from 
one to the next. As Figure 11.1 showed, language may 
be viewed as a double hierarchy, going from sound to 
meaning on the input side, and from meaning to vocal 
gestures in output. But we can repeat a nonsense sylla¬ 
ble like 'fronk' even though it has no particular mean¬ 
ing, and Figure 11.15 suggests that such recoding from 
sound to speaking may occur in the inferior parietal 
cortex. Broca's area was traditionally thought to serve 
the translation from thought to speech articulation, the 
current favored term is the left inferior frontal gyrus 
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FIGURE 11.18 Speaking may have evolved from socially evoked sound production. 
Vocalization has a dual-control pathway, much like breathing, facial expressions, eye move¬ 
ments, and other motor systems. The left branch is sometimes called the emotional motor path. 
Originating in prefrontal cortex, it follows a classic mammalian route for vocalization including 
the limbic system and peri-aqueductal gray (PAG). The PAG plays a major role in distress vocali¬ 
zations when rat pups and mothers are separated. Like the emotional motor path, the right-hand 
branch begins in prefrontal cortex, and then follows the better-known steps from Broca's area to 
motor cortex and thence to the cranial nerves for vocal control. Both pathways also receive input 
from basal ganglia and cerebellum. The right-hand pathway is under greater voluntary control. 
Source : Holstege et al., 2004. 


(L-IFG), because wider regions of the left hemisphere 
are thought to be involved. The older term 'Wernicke's 
area' for speech comprehension is now called the TPO 
junction (for temporal-parietal-occipital). This location 
makes sense, because we know that the temporal lobe 
is involved in memory-based concepts, including those 
abstracted from visual objects, like the differences 
between animate and inanimate figures, for tools and 
instruments, and for parts of the body. As Chapters 9 
and 10 pointed out, it may be that these very specific 
semantic regions serve to index larger classes of words 
and concepts, rather than being a localized semantic 
field. 


6* 1 A cultural treasury of words and ideas 

Language is used to communicate meanings. While pho¬ 
nemes are defined as minimal speech units that make a 
difference in meaning, words refer to things, which pho¬ 
nemes do not. Words are the basic building blocks of 
meaning. However, sentences, and particularly proposi¬ 
tions built out of sentences can be taken as basic 'mean¬ 
ing formulae', analogous to mathematical formulae. We 
think and communicate in sentences. Many communica¬ 
tions are of course elliptical, i.e. they abbreviate a whole 
thought into a few words. Nevertheless, they express a 
semantic proposition of some kind. 
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FIGURE 11.19 Speech production and perception loops con¬ 
stantly. Speech production and perception regions of the cortex are 
constantly exchanging information, both directly via subcortical 
connections, and indirectly, as we hear the sound of our own vocal 
apparatus. As pointed out in Chapter 2, there is good evidence that 
most people spend most of the day talking to themselves cortically. 
Indeed, studies of sleep stages seem to show that inner speech con¬ 
tinues during sleep and dreaming. Source : Dogil et al, 2002. 



FIGURE 11.20 In the motor homunculus (BA 5), muscular 
control of the mouth, jaw, tongue, vocal cords, as well as actions like 
chewing and swallowing reside next to Broca's area for the control 
of speaking (BA 6, 44, and 45). The 'motor homunculus' was first 
discovered by Wilder Penfield using electrical stimulation of motor 
cortex in awake patients during exploratory neurosurgery (Penfield 
and Roberts, 1959). Notice that mastication, vocalization, and swal¬ 
lowing are marked next to the mouth region of the homunculus. 
Source : Standring, 2005. 


As Chapter 10 suggested, the words of natural 
language are an immense legacy of useful chunks of 
meaning, developed over centuries. Indeed, in many 
cases we can trace word origins over more than a 
thousand years. A particularly nice example is the 
word 'quality' derived from the Latin translation (by 
Cicero) of the ancient Greek expression 'po io tes', or 
'what is it-ness'. Cicero apparently encountered it in 
a Socratic dialogue. Before his invented word qualitas 
became popular, European languages had no way to 
refer to the 'what is it-ness' of a sound, of a taste, or 
any other class of events. We can always use longer 
phrases for the ideas for which we do not have single 
words, but single words allow us to treat concepts as 
single chunks in our very limited working memory 
space. Thus, inventing new and useful words is a real 
contribution to our capacity to understand the world. 

Science is hard to imagine without a word like 
'quality'. Yet it took a certain realization at a certain 
point in intellectual history, more than twenty centu¬ 
ries ago, to chunk the concept of 'quality' in a way that 
we can use as an abstract noun. Probably all modern 



FIGURE 11.21 Brain activations for meaningful versus mean¬ 
ingless words. Words are not just sequences of phonemes. Although 
the brain basis of word meaning continues to be mysterious (pre¬ 
sumably because word meanings are part of long-term memory; see 
Chapter 9), meaningful words activate distinct regions of language- 
related cortex, when compared to nonsense words. In this study, 
three areas appeared in the comparison, the temporoparietal tran¬ 
sition area (tpTA), the posterior part of the medial temporal gyrus 
(pMTG), and the supramarginal gyrus (SMG). Source : Friederici and 
Kotz, 2003. 
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languages have some term for 'quality' today, because 
we can no longer think easily without it. Since modern 
languages have tens of thousands of words, with more 
being produced every day, we must multiply Cicero's 
invention by tens of thousands of other invented 
words to get some sense of the body of knowledge 
that is passed down to each new native speaker. 

Roget's Thesaurus , first compiled by Peter Mark Roget 
(1779-1869), is one historic effort to classify the words 
of a natural language into its semantic categories, so 
that one can find a word by starting with a concept. 
In a thesaurus, words with similar meanings are clus¬ 
tered together, giving us a kind of semantic treasury 
of standard ideas; indeed the word 'thesaurus' means 
'treasure'. Modern efforts along those lines use compu¬ 
tational methods, but characterizing our knowledge of 
basic concepts is still an awesome enterprise. George A. 
Miller's WordNet system at Princeton University is one 
effort along those lines (http://wordnet.princeton.edu/), 
an online lexical reference system. Over a period of 
years, it has classified more than 200,000 word-meaning 
pairs in English. Parallel efforts have been undertaken 
for other languages. By the WordNet count, English 
has some 128,000 single-meaning words, and about 
80,000 words with multiple meanings. However, this 
estimate does not include words with multiple syntac¬ 
tic roles, like 'book', which is both a noun and a verb 
(as in 'booking the arrested person'). 'Book' also can be 
adjectival (as in book-learning, book-binding, a bookish 
person, a book-reader, and so on.). 

Thus, the words and meanings that we know as 
skilled speakers of human language are enormous. 
The Oxford English Dictionary (OED) is another mas¬ 
sive effort along similar lines, tracing the history 
of standard English words to their written origins. 
Obviously, however, the real origins of words are lost 
in time, since language is basically a spoken medium, 
which only turned into a standardized printed body 
of knowledge long after the invention of writing, two 
or three millennia ago. Nevertheless, the OED gives 
us some sense of the sheer size of the human lexicon 
of words and concepts. 


6.2 Recognizing synonyms 

Gitelman et al. (2005) made a state-of-the-art effort 
to specify language areas using the identical cogni¬ 
tive task with the same group of subjects, looking at 
orthography (spelling), semantics, and phonology in 
the same experimental session. (Syntax involves word 
sequencing, and is therefore left until later.) Subjects 


were shown word pairs as in Figure 11.22, and asked 
to respond either 'same' or 'different'. In the case of 
'rain' versus 'reign', they were to say if the words 
were homophones (HOM), i.e. words with differ¬ 
ent spelling but the same pronunciation. In the case 
of 'boat' versus 'ship', they were to identify syno¬ 
nyms (SYN), different words with the same meaning. 
In the case of 'aunt' versus 'tuna', they were to iden¬ 
tify anagrams (scrambled letters, ANA); and in the 
control condition they were simply identifying 
whether two sets of four consonants were the same 
or different. The control condition allows common 
activity due to reading, visual stimulation, and the 
like to be subtracted from the other conditions. fMRI 
was recorded during stimulus exposure in each 
same-different trial. Comparisons were made within 
individuals to minimize inter-individual differences. 
Word stimuli were carefully selected for equal num¬ 
bers of nouns and verbs in comparison conditions, 
while other variables like word frequency were con¬ 
trolled statistically. 

Figure 11.23 shows the results for the HOM and 
SYN conditions. 

6.3 Current evidence about words and their 
meanings is fragmentary 

As Chapter 10 points out, we only know a few brain 
locations that are differentially sensitive to specific 
meanings, such as tools, animate versus inanimate 
objects, and the like. The function of those locations 
is still unclear. As Moscovitch (1992) has suggested, 
they could function as indices to a wide network of 
meaning-related connectivities in the neocortex. It 
is always possible that advances in brain recording 
methods will reveal a thesaurus-like array of neu¬ 
ronal patches, corresponding to semantic categories 


Phonologic Semantic Orthographic Control 

(HOM) (SYN) (ANA) (C) 


Match 

Rain 


Boat 


Aunt 


Fplk 

(response) 

Reign 


Ship 


Tuna 


Fplk 

Non-Match 

Axe 


Key 


Horse 


Gkjs 

(no response) 

Ask 


Lock 


Short 


Gskt 


FIGURE 11.22 Sample stimuli from the Gitelman et al. (2005) 
study. The experimental conditions are shown and include phono¬ 
logic (homophone, HOM), semantic (synonym, SYN), and ortho¬ 
graphic (anagram, ANA) contrasts in a match and a non-match 
condition. Source : Gitelman et al ., 2005. 
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FIGURE 11.23 Word meaning activation in the left hemi¬ 
spheres. Above, fMRI activity for matching homonyms, words with 
different spelling but the same pronunciations, like 'rain' and 'reign'. 
Below, matching synonyms, like 'boat' and 'ship'. A control activa¬ 
tion pattern elicited by meaningless consonant strings was subtracted 
from each of the fMRI activation patterns to eliminate brain activities 
related to reading and other common task features. Meaning-related 
activation (SYN, below) is far more widespread than homonym 
matching. Activity is prominent in the superior temporal gyrus and 
sulcus (STG and STS), and in the tip of the medial temporal gyrus 
(MTG). The inferior frontal gyrus (IFG) receives activation, as well as 
the lateral orbitofrontal gyrus (LOG). Higher in the left hemisphere 
for the SYN condition, there is some activation in the supplementary 
motor area (SMA), a region that is associated with the intention to act. 
The homonym task (HOM) also shows some activity in the posterior 
central sulcus (PCS). Source : Gitelman et al, 2005. 


(see Chapter 10). But it seems equally likely that word 
meanings are simply highly distributed, that they are 
inherently networks, and must therefore mobilize 
widely distributed regions of cortex. 



FIGURE 11.24 Syntactic tree structures allow nested proposi¬ 
tions. A basic syntactic tree, containing an embedded clause. Source : 
Gitelman et al., 2005. 


This is not to say that semantics lacks a specific 
brain basis. A number of recording methods show 
high sensitivity to distinctions between words that 
differ in meaning. So far, however, these meth¬ 
ods show very widespread changes in cortex (and 
elsewhere). 


7.0 SYNTAX, NESTING, AND 
SEQUENCING 

Syntax is often said to be the most distinctive aspect 
of language. Figure 11.24 shows a syntactic tree struc¬ 
ture with a subordinate clause. Tree structures can be 
considered to be recursive, i.e. they can embed sen¬ 
tences within sentences. Although for theoretical rea¬ 
sons it is often supposed that syntactic recursion can 
go to multiple levels, the human brain is limited to 
one or two self-embedded clauses. That is presum¬ 
ably because of the capacity limits of working mem¬ 
ory. When we develop more elaborate plans, as in the 
Tower of Hanoi, in chess playing, or in other complex 
activities, we typically use other memory aids. Thus, 
the sentence structure in Figure 11.24 is about typical 
for a single sentence. See Figure 11.25 for brain regions 
hypothesized to support syntactic processing. 

The tree structures of sentence syntax resemble the 
problem-solving spaces discussed in Chapter 10 (See 
Box 11.3). That is not likely to be an accident. In many 
ways, syntax gives us a framework for planning the 
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FIGURE 11.25 Syntax evokes distinctive brain regions. 
Grodzinsky and Friederici (2006) suggest that different cortical regions 
support different syntactic functions, as indicated in the figure. 
Source: Grodzinsky and Friederici, 2006. 

elements of a sentence in much the way we use goals 
and subgoals to plan a series of actions, like navigat¬ 
ing from one place to another, or making moves in a 
chess game. Syntax can therefore be seen as a cogni¬ 
tive planning tool, but it appears to be something 
human beings are adapted for. For skilled speakers, 
the planning of speech is of course mostly implicit 
(see Chapter 10). 

8.0 PROSODY AND MELODY 

Speaking and singing are similar activities; physically, 
singing is just a lengthening and tuning of vowel 
sounds. If you stretch out the vowels of any word - like 
'cognitive neuroscience' - you are already singing in 
a monotone. Vary the pitch of each syllable, and you 
have a little song. Like language, music and dance are 
species-specific capacities for humans. (The biologi¬ 
cal basis for that is puzzling: language has an obvious 
survival function, but why music?) 

Some sort of musical scale is widely used among 
many different cultures. All divide up the octave. In 
Western music, major scales are commonly perceived 
to be happier than minor scales. Musical notes, like 
the vowels of language, are based on the physics of 
resonant tubes, like bamboo flutes, or the tension of 
a bow string. When the vibrating column of air in a 
flute is half as long, it sounds an octave higher. When 
a guitar string is twice as tight, it is also heard as the 


same note an octave up. Small animals make high- 
pitched sounds and big ones low ones because of the 
acoustics of resonant cavities. Thus, there is a physi¬ 
cal basis for sound perception with biological implica¬ 
tions. It makes more sense to run away if you hear the 
roar of a lion than the chirp of a cricket. 

Human speech also conveys emotion via intonation 
and sound quality. Depressed people tend to show a 
declining intonation contour, perhaps reflecting a lower 
level of subglottal air pressure. Joy is often signalled 
with upward inflections of sound. Emotional expres¬ 
sion has musical intervals. Dividing up the octave is 
universal, though different musical cultures do it dif¬ 
ferently. Emotional expression and signaling may pre¬ 
cede and later co-evolve with denotative language. 
Different musical intervals - i.e. two-tone sequences - 
seem to have different emotional meanings. 

Even rhythm is an aspect of normal speech, as 
we can see in the stress pattern of spoken sentences: 
The rain/ in Spain/ falls mainly/ in the plain/'. You 
can intuitively mark the stressed syllables in any sen¬ 
tence. Even languages that minimize syllable stress, 
like French, still give a melodic shape to phrases. Babies 
babble in singsong, and adults spontaneously speak 
to young children (and pets) using exaggerated into¬ 
nation. While different cultures have different musi¬ 
cal forms, it seems likely that there are some universal 
(probably biological) connections between language 
and music. 


9.0 MEANINGFUL STATEMENTS 

We do not speak in single words, but in propositions - i.e. 
in semantically meaningful statements about the 
world. Syntax is tailored to enable such propositions, 
but even when syntax is impaired, as in some types of 
aphasias, people may still be able to think and express 
themselves propositionally. For example, aphasics 
may be able to point to desired food, to express dis¬ 
comfort or pleasure, or express any number of 'para- 
linguistic' statements about themselves and the world. 
Shastri (2002) has proposed a model of meaningful 
propositions encoded in a simulated neural network, 
based on hippocampal and cortical neurons. Figure 
11.26 gives an idea of how different neurons might 
express different parts of a proposition. By coordinat¬ 
ing the firing of these neurons (for example, by way of 
gamma synchrony), one can suggest how full proposi¬ 
tions might be expressed in the brain. 
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BOX 11.3 Some basics of syntax 



The building blocks of syntactic knowledge 

Operation Description Examples 


Basic relationships among words and phrases 

LEX Lexical relations that have syntactic relevance. 

For example, an argument structure of a 
verb - the types and numbers of arguments 
that natural language predicates require. 

MERGE A class of highly constrained structure¬ 

building operations, which analyze sentences 
into hierarchical structures. This example 
shows how syntactic MERGE rules build a 
sentence from the set of lexical categories 
('numeration'). MERGE creates phrasal 
nodes (NP = noun phrase, VP = verb 
phrase, PP = prepositional phrase) out of 
merged categories (DETerminer Noun, Verb, 
Preposition), which are in turn merged into a 
'root', sentence node. 


Dependency relations within a sentence 

MOVE X p A central syntactic operation on trees 

(created by MERGE). It links an audible 
phrase XP (=NP, VP, PP) to one or more 
silent, yet syntactically active, position(s) 
'■'in the representation of the same 
sentence. 

MOVEy A movement relationship that links 

distinct positions a verb might occupy. 
Only one is audible; the rest are silent 
(' '). This relation is shown in English 
yes/no questions, and in German, in 
which the verb 'sah' (saw) and its 
participle 'gesehen' (seen) occupy 
different positions. 

BIND A relationship that determines how 

reflexives and pronouns link to other 
NPs, on which they depend for reference, 
in the same sentence. 


1. argument: He ran/slept/died 

2. arguments: He saw/hit/followed Mary 

3. arguments: He gave/sent/mailed Sue presents 


Numeration: 

(DET = a, the; N = man, woman, tree; V = saw; P = near) 
Result of Iterated MERGE: 



Sam knows that the saw the ballet dancer on Monday 

=> 

Which dancer does Sam know that he saw ■ on 
Monday? 


English: John is tall => is John + tall? 

German: Hans hat Maria gesehen => 

Hans sah Maria + => Gestem Sah Hans 4 Maria 4 


John looked at himself 
Mary asked John to help her 
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FIGURE 11.26 A proposition may involve a 
distributed brain network. At the level of neural net¬ 
works, the proposition, "Soccer moms are likely to own 
minivans' may be represented by a collection of neural 
populations corresponding to the main elements of the 
proposition: minivans, moms, soccer, relationships like 
'owner-of', and the like. Such propositional networks 
have not been observed in the brain. The model simply 
suggests neural nets as we understand them. Source : 
Shastri, 2002. 


10.0 UNIFIED REPRESENTATIONS 
OF LANGUAGE 

In the 19th century, the psychologist Wilhelm Wundt 
suggested that language production begins with a 
Gesamtvorstellung, a unified mental representation of 
a sentence one was about to utter (Blumenthal, 1979). 
This unified representation would need many differ¬ 
ent levels of description: semantic, syntactic, phone¬ 
mic, perhaps vocal, pragmatic (i.e. involving one's 
goals), and more. Once this many-layered tower of 
brain activations was ready, it would begin issuing 
vocal commands to move the parallel structure of the 
unified representation into a long series of vocal ges¬ 
tures. Figure 11.27 shows a contemporary example 
of the same basic idea. Notice that there are many 
simultaneous or overlapping levels of description and 


control, each presumably reflected in something like a 
set of neural arrays. In this case, each level is associated 
with a distinctive signal in the evoked potential 
(Friederici, 2002). For example, semantic information 
is known to evoke a large negative wave 400 ms after 
the onset of a stimulus (called N400). 

If that seems awesomely complex, that is probably 
accurate. It is always useful to remember that language as 
we know it appears to be a distinctively human 
achievement. The human lineage separated from other 
hominids some 3 million years ago, and it is often sug¬ 
gested that the basic elements of human intelligence, 
such as language, persistence in problem-solving, coop¬ 
erative and competitive behavior, and the like, may 
have emerged on the order of 100,000 years ago. While 
our vocabulary, for example, is the result of centuries of 
cultural development, the biological preparedness of 
our brains and bodies for such developments has many 
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Working memory Online processes 



FIGURE 11.27 Putting it all together. A neurocognitive model of auditory sentence processing. The boxes represent the functional pro¬ 
cesses; the ellipses the underlying neural correlate identified either by fMRI, PET, or ERPs. The neuroanatomical specification (indicated by text 
in parentheses) is based on either fMRI or PET data. The ERP components specified in their temporal structure (left-hand side) are assigned to 
their neural correlation by the function rather than the localization of their generator. Abbreviations: BA, Brodmann's area; ELAN, early left- 
anterior negativity; ERP, event-related brain potential; fMRI, functional magnetic resonance imaging; IFG, inferior frontal gyrus; MTG, middle 
temporal gyrus; MTL, middle temporal lobe; PET, positron imaging tomography; STG, superior temporal gyrus. Source : Friederici, 2002. 


evolutionary layers, some shared with other mam¬ 
mals, others with land-dwelling vertebrates. The kind 
of complexity we see displayed in Figure 11.27 evolved 
over many generations. 

Chapters 3 and 6 showed that the visual system 
appears to have at least one region of integration 'where 
everything comes together', the inferotemporal cortex 
(area IT) (Sheinberg and Logothetis, 1997). In that area, 
neurons respond not to single retinal stimuli, nor to 
separate features like colors or light edges, but rather to 
entire visual objects. It is at least possible that language 
may have a similar region of integration (Hagoort, 2005; 
Figure 11.28). 

The notion of a place 'where everything comes 
together' for some brain function is attractive and can 
be modeled in neural networks (e.g. McClelland, 1986; 
Shanahan, 2005) as well as in symbolic models of cog¬ 
nition (Baars and Franklin, 2003). It is an ancient idea, 
going back at least to Aristotle, who proposed that 
human cognition must have a way to combine infor¬ 
mation from the special senses like vision and hear¬ 
ing into single multimodel objects in the world - the 


guitar that you can see, touch, and listen to (Baars, 
1988). Aristotle suggested that there must be a 'com¬ 
mon sense' - some central sensory modality that com¬ 
bines all the special senses into a single medium. That 
is indeed one plausible account of multimodal regions 
of the parietal and frontal lobes. However, many brain 
regions show hub-like neuronal connections, ranging 
from the brainstem reticular formation to the amy¬ 
gdala, entorhinal cortex, prefrontal lobes, and thala¬ 
mus. Some theorists suggest that the entire cortex, 
or the thalamo-cortical system, should be viewed as 
massive networks for integrating, differentiating, and 
distributing signals (e.g. Edelman, 1989; Edelman and 
Tononi, 2000; Freeman, 2004). One can imagine a brain 
consisting of hubs of hubs, consistent with some of 
the concepts of consciousness discussed in Chapter 8. 
Hagoort's notion of the left IFG as one anatomical hub 
for integrating language and meaning appears to be 
one version of this general idea. However, as pointed 
out above, attractive and plausible ideas do not con¬ 
stitute proof, and this hypothesis requires additional 
evidence. 
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FIGURE 11.28 An area of integration? Hagoort (2005) points 
out that Broca's area has multiple functions which go beyond the 
control of speech. Nevertheless, he suggests that an expanded ver¬ 
sion of Broca's area may be considered an area for unification of 
speech and semantic information. To avoid confusion with the tra¬ 
ditional concept of Broca's area, this part of cortex is referred to by 
its location as the left inferior frontal gyrus (L-IFG). Source: Hagoort, 
2005, Figure 5. 

11.0 SUMMARY 

Language is a distinctive human capacity, one that 
makes it possible to transmit culture across time and 
space. The brain bases of language are still being clari¬ 
fied, but the discoveries of 19th century neurology 
continue to be important starting points. Broca's area 


for speaking and Wernicke's for speech comprehen¬ 
sion are only part of large cortical regions involved. 
Current work has expanded and fractionated the tra¬ 
ditional language areas, so that the left inferior frontal 
gyrus (L-IFG) is a more appropriate term for Broca's 
area, and posterior auditory and speech regions of the 
parietal and temporal cortex are more accurate than 
the term 'Wernicke's area'. However, there is constant 
interplay between frontal and posterior language 
areas, and a hard-and-fast division is to some extent 
artificial. In addition, the evidence is strong that the 
right hemisphere has its own role to play in language 
perception. 

Each level of linguistic description has neuroimag¬ 
ing evidence in its favor (see Figure 11.1). Nonetheless, 
we do not have the kind of detailed knowledge of lan¬ 
guage cortex that we have for vision. One major rea¬ 
son is the absence of an appropriate animal model, 
like the macaque monkey for vision. However, recent 
techniques that are appropriate for humans are begin¬ 
ning to approach the right level of detail, and we 
may soon find such things as speech feature sensitive 
arrays of neurons, for both speech input and output. 

So much of speech and language is dependent on 
long-term memory, however, that many scientists 
believe that we must ultimately look for highly distrib¬ 
uted cortical networks to account for the vocabulary, 
syntax, and semantics of language. These networks are 
believed to depend upon the synaptic connectivities 
of very large numbers of neurons. Current techniques 
are just beginning to be able to assess such web-like 
patterns of distributed neurons. 


12*0 PRACTICE DRAWINGS AND 
STUDY QUESTIONS 

1 Fill in the labels in Figure 11.29 of processing 
hierarchies of language. 

2 Give examples of the need for 'top-down' or 
'expectation-driven' processing in the input 
flow of speech. What about the output flow? 

3 What evidence is there for biological prepared¬ 
ness of human language capacity? 

4 In what respects does the human vocal appara¬ 
tus resemble a musical instrument? 

5 In Figure 11.30, show the following: 

a A cortical region likely to be involved in 
speech perception 


b One for speech planning 
c One for motor control of the vocal tract, such 
as the tongue 

d One for coordinating perception and produc¬ 
tion (you may have to draw it in). 

6 What role, if any, does the right hemisphere play 
in language? 

7 What is the approximate size of the lexicon 
(vocabulary of natural language)? 

What do we know about the organization of this 
number of words and meaning? 

8 Explain a syntactic ambiguity. What do ambi¬ 
guities imply about the nature of language 
processing? 
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Levels of language processing 



FIGURE 11.29 Linguistic processing hierarchy. 



FIGURE 11.30 
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He is fitful , irreverent , indulging at times in the grossest profanity (which was not previously his 
custom), manifesting but little deference for his fellows , impatient of restraint or advice when 
it conflicts with his desires . ... A child in his intellectual capacity and manifestations , he has 
the animal passions of a strong man . . . . His mind was radically changed , so decidedly that his 
friends and acquaintances said he was ( no longer Gage . 

Harlow JM. Recovery from the passage of an iron bar through the head. 

Publ Mass Med Soc 1868; 2: 327-347. Quoted in: 
BMJ 1998; 317: 1673-1674 (19 December) 
( No longer Gage’: an iron bar through the head . 



A reconstruction of the injury to Phineas Gage. A recon¬ 
struction of Phineas Gage's railroad accident in 1848, 
when he was 25 years old. Notice likely damage to 
orbitofrontal and medial frontal regions as well. Injuries 
like this create damage from swellings, bleeding, heat, 
infection, inflammation, and physical twisting of tissues 
that extend far beyond the immediate region of impact. 
Thus, we do not really know the extent of brain damage 
in this classic neurological patient. Source : Squire et al., 
2003. 
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1.0 INTRODUCTION 

What do these behaviors have in common? Remem¬ 
bering your friend's new cell phone number while 
looking for a piece of paper. Deciding to study first 
and play basketball second. Planning a new route 
when a road has been closed for repair. Paying atten¬ 
tion while you are reading this sentence. Changing 
your mind about raising the stakes in a poker game. 
All of these behaviors - and many more - are guided 
by the frontal lobes. 


1.1 The many and complex frontal lobe 
functions 

The frontal lobes have been described as a con¬ 
trol center for functions such as paying attention 
selectively to one item rather than another, making 
plans and revising them when needed, monitoring 
the world around us - complex functions that are 
part of our everyday life. The frontal lobes also are 
described as the 'action lobes' where physical actions 
are planned and motor system activity is initiated. 
The frontal lobes also are described as the 'home' 
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FIGURE 12.1 The major divi¬ 
sions of the prefrontal cortex. Pre¬ 
frontal cortex can be divided into 
lateral (side), medial (midline), ven¬ 
tral (bottom), and frontal regions. 
The lateral division divides into 
upper (dorsal) and lower (ventral) 
halves, separated by a major hori¬ 
zontal fold, the inferior lateral sulcus. 
Source: Simons and Spiers, 2003. 


of our personality and social morality. How does 
one brain region provide these diverse and complex 
functions? 

The answer to this question is still being investi¬ 
gated, but we have learned about frontal lobe func¬ 
tion through many investigative pathways. In animal 
research, we have learned about the role of the fron¬ 
tal lobe using single unit recordings. In humans, we 
have learned about the many roles of the frontal lobe 
through brain damage and disease. A key figure in 
our knowledge about frontal lobe function is Phineas 
Gage, the railroad worker who had major frontal lobe 
damage due to a railroad accident. Phineas' sudden 
personality change following the accident shed light 
on the role of frontal regions in personality formation 
and in social cognition. Studies of people who have 
had frontal lobe damage through disease, brain injury, 
or disorder have aided us in developing categories of 
frontal lobe function based on where the brain dam¬ 
age or disease has occurred and the resultant change 
in behavior. More recently, neuroimaging studies of 
healthy individuals have provided new information 
about the localization of patterns of frontal lobe activ¬ 
ity occurring during tasks that tap frontal lobe func¬ 
tions, such as voluntary or executive attention and 
decision-making. 

The frontal lobes are a vast mosaic of cell types and 
cortical regions, diverse in their cell structures, ana¬ 
tomical features, and connectivity patterns. Unlike 
regions in sensory cortex, the frontal lobes do not 
have a single job to do - they are not specialized for 
decoding speech sounds or recognizing faces. Rather, 
the frontal lobes are engaged in almost all aspects of 
human cognitive function. 


In this chapter, we will present results of neuroim¬ 
aging studies that have provided new data on where, 
what, and how specific regions in the frontal lobes are 
activated during cognitive tasks. Next, we will look at 
frontal lobe syndromes that are observed in patients 
with damage to the frontal lobe and connected 
regions. Finally, we will see how the results of neu¬ 
roimaging and patient studies combine to inform us 
about the role of the frontal lobes in human behavior. 


1.2 From the silent lobes to the organ 
of civilization 

It took scientists many years to begin to appreciate 
the importance of the frontal lobes for cognition. But 
when this finally happened, a picture of particular 
complexity and elegance emerged. For the most part, 
this chapter will focus on the prefrontal cortex (PFC), 
the most anterior part of the frontal lobes, in front 
of the motor areas. There are many subregions in the 
PFC, but four regions most typically are identified 
when assessing their functional role in cognition: dor¬ 
solateral PFC (DLPFC), ventrolateral PFC (VLPFC), ante¬ 
rior PFC (APFC), and medial PFC (MPFC) (Figure 12.1). 

PFC is located in front of the primary motor cortex, 
sometimes called the motor strip. The frontal lobes 
used to be known as 'the silent lobes' because they are 
not easily linked to any single, easily defined function. 
Over the last decades, however, new imaging tech¬ 
niques are suggesting more specialized regions within 
the PFC for functions like executive control, conflict 
monitoring, emotion, and working memory. There is 
still considerable debate about these issues, however. 
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FIGURE 12.2 The prefrontal cortex 
expands over mammalian and primate evo¬ 
lution. A greatly enlarged prefrontal cortex 
is a distinctively human and primate feature. 
Other large-brained mammals like whales and 
dolphins have expanded parietal rather than 
prefrontal regions. Bottom right, a human 
brain, with a chimp brain on the bottom left. 
Source : Squire et al., 2003. 



2*0 PHYLOGENY AND ONTOGENY 

If it seems to you that the role the frontal lobes play 
in cognition seems uniquely human, you are right! 
The vast expansion of the frontal lobes during evolu¬ 
tion and their maturational path during the lifetime in 
humans are unique among living creatures. 


2*1 Phylogeny 

In evolution, the frontal lobes accelerated in size only 
with the great apes. These regions of cortex under¬ 
went an explosive expansion at the late stage of evo¬ 
lution. According to Brodmann (1909), the PFC or its 


analogs account for 29% of the total cortex in humans, 
17% in the chimpanzee, 11.5% in the gibbon and the 
macaque, 8.5% in the lemur, 7% in the dog, and 3.5% 
in the cat (Figure 12.2). While whales and dolphins 
have large brains, it is the parietal rather than frontal 
cortex that has expanded in these aquatic mammals. 

2*2 Ontogeny 

As the seat of goals, foresight, and planning, the fron¬ 
tal lobes are perhaps the most uniquely human of 
all the components of the human brain. In 1928, the 
neurologist Tilney suggested that all human evolu¬ 
tion should be considered the 'age of the frontal lobe', 
but scientific interest in the PFC was late in coming. 
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TABLE 12.1 Some common prefrontal functions 

1 Planning, setting goals, and initiating action 

2 Monitoring outcomes and adapting to errors 

3 Mental effort in pursuing difficult goals 

4 Interacting with other regions in pursuit of goals (basal ganglia, 
thalamic nuclei, cerebellum, motor cortex) 

5 Having motivation, being willing to engage in action 

6 Initiating speech and visual imagery 

7 Recognizing others people's goals, engaging in social cooperation 
and competition 

8 Regulating emotional impulses 

9 Feeling emotions 

10 Storing and updating working memory 

11 Active thinking 

12 Enabling conscious experiences (Deheane, 2001) 

13 Sustained attention in the face of distraction 

14 Decision-making, switching attention and changing strategies 

15 Planning and sequencing actions 

16 Unifying the sound, syntax, and meaning of language 

17 Resolving competition between plans 


Only gradually did it begin to reveal its secrets to the 
great scientists and clinicians like Hughlings Jackson 
(1884) and Alexander Luria (1966), and in the last few 
decades to researchers like Antonio Damasio (1993), 
Joaquin Fuster (1997), Patricia Goldman-Rakic (1987), 
Donald Stuss and Frank Benson (1986), and others. 


3*0 FUNCTION OVERVIEW 

If the frontal lobes are the 'organ of civilization', then 
what exactly is their function? What is their 'civiliz¬ 
ing' effect? The functions of the frontal lobes defy 
a simple definition. They are not invested with any 
single, ready-to-label function. A patient with frontal- 
lobe damage will typically retain the ability to move 
around, use language, recognize objects, and even 
memorize information. However, PFC plays the cen¬ 
tral role in forming goals and objectives and then in 
devising plans of action required to attain these goals. 
It selects the cognitive skills needed to implement the 
plans, coordinates these skills, and applies them in a 
correct order. Finally, the PFC is responsible for evalu¬ 
ating our actions as success or failure relative to our 
intentions (Table 12.1). 

3*1 ‘Memory of the future* 

David Ingvar (1985) coined the phrase: 'Memory of 
the future'. Ingvar was referring to one of the most 


important functions of advanced organisms: making 
plans and then following the plans to guide behavior. 
Unlike primitive organisms, humans are active, rather 
than reactive, beings. The transition from mostly re¬ 
active to mostly proactive behavior is among the cen¬ 
tral themes of the evolution of the nervous system. We 
are able to form goals, our visions of the future. Then 
we act according to our goals. But, in order to guide 
our behavior in a sustained fashion, these mental 
images of the future must become the content of our 
memory; thus the 'memories of the future' are formed. 

Human cognition is forward-looking, proactive 
rather than reactive. It is driven by goals, plans, hopes, 
ambitions, and dreams, all of which pertain to the 
future and not to the past. These cognitive powers 
depend on the frontal lobes and evolve with them. 
The frontal lobes endow the organism with the abil¬ 
ity to create neural models as a prerequisite for mak¬ 
ing things happen, models of something that, as of 
yet, does not exist but which you want to bring into 
existence. 

To makes plans for the future, the brain must have 
an ability to take certain elements of prior experiences 
and reconfigure them in a way that does not copy 
any actual past experience or present reality exactly. 
To accomplish that, the organism must go beyond 
the mere ability to form internal representations, the 
models of the world outside. It must acquire the 
ability to manipulate and transform these models. We 
can argue that tool-making, one of the fundamen¬ 
tal distinguishing features of primate cognition, 
depends on this ability, since a tool does not exist in 
a ready-made form in the natural environment and 
has to be imagined in order to be made. The neural 
machinery for creating and holding 'images of the 
future' was a necessary prerequisite for tool-making, 
and thus for launching human civilization. 

We can also argue that the generative power of lan¬ 
guage to create new ideas depends on this ability as 
well. The ability to manipulate and recombine inter¬ 
nal representations depends critically on the PFC, 
which probably made it critical for the development 
of language. 


3.2 Self-awareness and executive function 

Goal formation is about 'I need' and not about 'it is'. 
Therefore, the ability to formulate goals must have been 
inexorably linked to the emergence of the mental rep¬ 
resentation of self. It should come as no surprise that 
self-awareness is also intricately linked to the frontal lobes. 
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All these functions can be thought of as metacog- 
nitive rather than cognitive, since they do not refer to 
any particular mental skill, but provide an overarch¬ 
ing organization for all of them. For this reason, some 
authors refer to the functions of the frontal lobes as 
executive functions, by analogy with a governmental or 
corporate executive. 

4*0 CLOSER LOOK AT FRONTAL LOBES 

As we have mentioned, the neuroanatomy and neu¬ 
rophysiology of the frontal lobes reflect their diverse 
and complex functions, with many distinct regions 
that differ sharply in their cellular and anatomical 
structure. Adding another layer of complexity is their 
high degree of connectivity across many brain regions. 
In this section, we highlight the main subdivisions of 
the frontal lobes and show the amazing neural high¬ 
ways that connect them to the rest of the brain. 

4*1 Neuroanatomy and neurophysiology 
of the frontal lobes 

The boundaries of the frontal lobes typically are 
described using gross anatomical structures and 
Brodmann areas to define boundaries of frontal lobe 
regions. On the surface of cerebral cortex the following 
boundaries are used to delimit the frontal lobes: on the 
dorsolateral surface of the brain they lie just in front of 
the demarcation line formed by the lateral (or Sylvian ) 
fissure and the central sulcus ; medially, the boundaries 
are roughly formed by the continuation of the central 
sulcus to the mediodorsal surface of the brain, and by 
an imaginary line corresponding to a projection of the 
lateral fissure to the medioventral surface of the brain 
(Carpenter and Parent, 1995). 

4*2 How prefrontal cortex is defined 

A more precise definition of PFC can be accom¬ 
plished by using Brodmann area maps (Brodmann, 
1909). Brodmann areas are based on the types of neu¬ 
rons and connections that typically are found there 
within. According to this definition, the PFC consists 
of Brodmann areas 8, 9, 10, 11, 12, 13, 44, 45, 46, and 
47 (Fuster, 1997; Figure 12.3). These areas are charac¬ 
terized by the predominance of the so-called granular 
neural cells found mostly in layer IV (Campbell, 1905, 
in Fuster, 1997). 



FIGURE 12.3 Brodmann areas in the frontal lobes. Areas for¬ 
ward of motor cortex are considered to be prefrontal. (Brodmann 
areas 4 and 6 are motor and premotor regions.) However, the bound¬ 
ary is not rigid. It is often useful to think of a gradual transition 
between more "cognitive 7 areas and primary motor cortex (BA 4), 
which directly controls voluntary muscles. Source : Adapted by 
Bernard J. Baars from M. Dubin, with permission; drawn by Shawn Fu. 


Another method of outlining the PFC is through 
its subcortical projections. The dorsomedial thalamic 
nucleus is, in a sense, the point of convergence, the 
'summit' of the integration occurring within the spe¬ 
cific thalamic nuclei (Figure 12.4). PFC is then defined 
as the area receiving projections from the dorsomedial 
thalamic nucleus. 

Finally, the PFC sometimes is delineated through 
its biochemical pathways. According to that definition, 
the PFC is defined as the area receiving projections 
from the mesocortical dopamine system. The various 
methods of delineating the PFC outline roughly simi¬ 
lar territories. 
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FIGURE 12.4 Prefrontal cortex is also defined by major thalamic connections. The yellow cortical 
areas are considered to be prefrontal because of their connections with mediodorsal nucleus of the thalamus. 
Many scientists believe that the connections between cortex and thalamus are so close and intimate that tha¬ 
lamic nuclei can be considered to be additional layers of cortex. Thus, we are looking at a thalamo-cortical 
system. Source : Standring, 2005. 


43 The vast connective highways of the 
frontal lobes 

The PFC is connected directly with every distinct 
functional unit of the brain (Nauta, 1972). It is con¬ 
nected to the highest levels of perceptual integra¬ 
tion, and also with the premotor cortex, basal ganglia, 
and the cerebellum, all involved in aspects of motor 
control and movements. PFC also is connected with 
the dorsomedial thalamic nucleus, often considered to 
be the highest level of integration within the thala¬ 
mus; with the hippocampi and medial temporal struc¬ 
tures, known to be critical for memory; and with the 
cingulate cortex, believed to be critical for emotion 
and dealing with uncertainty. In addition, PFC con¬ 
nects with the amygdala, which regulates most emo¬ 
tions and social cognition, and with the hypothalamus, 
in charge of control over the vital homeostatic func¬ 
tions of the body. Finally, PFC is also well connected 
with the brainstem nuclei involved in wakefulness. 


arousal, and overall alertness, regulation of sleep and 
REM dreams. A schematic (and by no means exhaus¬ 
tive) representation of this complex connectivity is 
depicted in Figure 12.5. 

This unique connectivity makes the frontal lobes 
singularly suited for coordinating and integrating 
the work of other brain structures (Figure 12.6). This 
extreme connectivity also puts the frontal lobes at a 
particular risk for disease. Some scientists believe that 
the PFC contains a map of the whole cortex, an asser¬ 
tion first made by Hughlings Jackson (1884) at the end 
of the nineteenth century. This hypothesis asserts that 
prefrontal regions are needed for normal conscious¬ 
ness. Since any aspect of our mental world may, in 
principle, be the focus of our consciousness, it stands 
to reason that an area of convergence of all its neural 
substrates must exist. This leads to the provocative 
proposition that the evolution of consciousness, the 
highest expression of the developed brain, parallels 
the evolution of the PFC. 
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FIGURE 12.5 Connectivity of the prefrontal cortex with struc¬ 
tures involved in motor function. The three lateral frontal regions of 
the motor hierarchy (prefrontal, premotor, motor cortices) are inter¬ 
connected with thalamus, basal ganglia, and cerebellum by recurrent 
axonal loops that are essential to motor control. Abbreviations: CM, 
central median nucleus; MD, mediodorsalis nucleus; VA, anteroventral 
nucleus; VL, ventrolateral nucleus. Source : Fuster, 2008. 


5.0 A CLOSER LOOK AT FRONTAL LOBE 
FUNCTION 


5.1 Executive functions 

As mentioned earlier, currently, the concept of execu¬ 
tive functions is inextricably linked to the function of 
the frontal lobes. The groundwork for defining the 
executive functions system was laid by Alexander 
Luria as early as 1966 (Luria, 1966). At the time, Luria 
proposed the existence of a system in charge of inten- 
tionality, the formulation of goals, the plans of action 
subordinate to the goals, the identification of goal- 
appropriate cognitive routines, the sequential access 
to these routines, the temporally ordered transition 



FIGURE 12.6 A. Midsagittal view of the fiber bundles extend¬ 
ing into the prefrontal cortex, shown in green, and into the premo¬ 
tor and motor cortex, shown in blue. B. An axial view of the same 
fiber bundles. Source : Hofer & Frahm, 2006. 


from one routine to another, and the editorial evalua¬ 
tion of the outcome of our actions (Figure 12.7). 

Subsequently, two broad types of cognitive opera¬ 
tions linked to the executive system figured most 
prominently in the literature: 

1 An organism's ability to guide its behavior by 
internal representations (Goldman-Rakic, 1987) - 
the formulation of plans and then guiding behav¬ 
ior according to these plans 

2 An organism's ability not only to guide its behavior 
by internal representations, but also the capacity 
of 'switching gears' when something unexpected 
happens (Milner, 1982). 

To deal effectively with such transitions, a particu¬ 
lar ability is needed - mental flexibility - that is the 
capacity to respond rapidly to unanticipated environ¬ 
mental contingencies. Sometimes this is referred to as 
an ability to shift cognitive set. Additionally, the execu¬ 
tive system is critical for planning and the generative 
processes (Goldberg, 2001). 
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FIGURE 12.7 Prefrontal lobes coordinate goals and actions. 
An integrated model of goal processing in the prefronal cortex (the 
dorsolateral part), connecting to the anterior cingulate to resolve 
conflicts. Memory is involved in retrieving stored action plans and 
contextual information (temporal lobe and MTL), and the posterior 
parietal cortex is considered as a control of spatial attention, needed 
for actions. The thalamus serves as a hub for multiple functions. 
Source : Schneider and Chein, 2003. 


Fuster (1997) enlarged on the premise originally 
developed by Luria by suggesting that the so-called 
executive systems can be considered functionally 
homogeneous in the sense that they are in charge of 
actions, both external and internal (such as logical 
reasoning). In general, the executive functions are not 
unique to humans. However, the uniqueness of the 
human executive functions is in the extent to which they 
are capable of integrating such factors as time, informa¬ 
tional novelty, and complexity, and possibly ambiguity. 


5.2 Social maturity and moral development 

While many neuroimaging and behavioral studies 
have investigated attention, working memory, and 
executive control processes in the PFC, the frontal lobes 
also play a critical role in the development of social 
cognition - a key link to the role of the frontal lobes as 
the 'organ of civilization'. The capacity for volitional 
control over one's actions is not innate, but it emerges 
gradually through development and is an important, 
perhaps central, ingredient of social maturity. 

5.2.1 Early life experience and orbitofrontal 
cortex development 


of the orbitofrontal cortex during the first months of 
life. By contrast, early-life stressful experiences perma¬ 
nently damage the orbitofrontal cortex, predisposing 
the individual to later-life psychiatric diseases. This 
implies that early social interactions help shape the 
brain. Scientists have known for years that early sen¬ 
sory stimulation promotes the development of visual 
cortex in the occipital lobes, and early-life sensory 
deprivation retards its development. It is possible that 
social stimulation is to the development of the frontal 
cortex what visual stimulation is to the development 
of the occipital cortex. 

5.2.2 Moral development and the frontal cortex 

Furthermore, following this logic, is it possible that 
moral development involves the frontal cortex, just as 
visual development involves occipital cortex and lan¬ 
guage development involves temporal cortex? The PFC 
is the association cortex of the frontal lobes, the 'action 
lobes'. The posterior association cortex encodes generic 
information about the outside world. It contains the 
taxonomy of the various things known to exist and 
helps recognize a particular exemplar as a member of a 
known category. By analogy, the PFC may contain the 
taxonomy of all the sanctioned , moral actions and behav¬ 
iors. And could it be that, just as damage or maldevel- 
opment of the posterior association cortex produces 
object agnosias , so does damage or maldevelopment of 
the PFC produce, in some sense, moral agnosia ? 

A report by Damasio and colleagues lends some 
support to this idea (Anderson et al. f 1999). Damasio 
studied two young adults, a man and a woman, 
who suffered damage to the frontal lobes very early 
in life. Both engaged in antisocial behaviors: lying, 
petty thievery, truancy. Damasio claims that not only 
did these patients fail to act according to the proper, 
socially sanctioned moral precepts, but they even 
failed to recognize them as morally wrong. 

The orbitofrontal cortex is not the only part of the 
frontal lobes linked to socially mature behavior. The 
anterior cingulate cortex occupies a mid-frontal position 
and is closely linked to the PFC. The anterior cingu¬ 
late cortex traditionally has been linked to emotion. 
According to Michael Posner, it also plays a role in 
social development by regulating distress (Posner and 
Rothbart, 1998). 

5.2.3 Age of maturity and frontal lobe development 


Allan Schore (1999) believes that early mother-infant The implicit definition of social maturity changes 

interaction is important for the normal development throughout the history of society, and so does the 
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time of 'coming of age'. In modern Western societies, 
the age of 18 (or thereabout) has been codified in the 
law as the age of social maturity. This is the age when 
a person can vote and is held responsible for his or 
her actions as an adult. The age of 18 is also the age 
when the maturation of the frontal lobes is relatively 
complete. Various estimates can be used to measure 
the course of maturation of various brain structures. 
Among the most commonly used such measures is 
pathway myelinization (Yakolev and Lecours, 1967). 
The frontal lobes cannot fully assume this role until 
the pathways connecting the frontal lobes with the 
far-flung structures of the brain are fully myelinated. 

The agreement between the age of relatively com¬ 
plete maturation of the frontal lobes and the age of 
social maturity is probably more than coinciden¬ 
tal. Without the explicit benefit of neuroscience, but 
through cumulative everyday common sense, society 
recognizes that an individual assumes adequate con¬ 
trol over his or her impulses, drives, and desires only 
by a certain age. Until that age, an individual cannot 
be held fully responsible for his or her actions in either 
a legal or moral sense. This ability appears to depend 
critically on the maturity and functional integrity of 
the frontal lobes. 


6*0 NEUROIMAGING THE EXECUTIVE 
BRAIN 

The two broad types of cognitive operations linked 
to the executive systems in the frontal lobe have been 
extensively investigated using functional neuroimag¬ 
ing techniques such as PET and fMRI. We summarize 
some general findings here, separating the results into 
three sections: attention and perception, working memory, 
executive function and motor control. But first, a word 
of caution about functional neuroimaging studies of 
complex processes: as we have described, the PFC is 
intricately involved in many cognitive and executive 
processes such as paying attention, holding something 
in mind for a few moments, switching attention when 
needed, and making decisions. Thus, any task used in 
a neuroimaging study will necessarily involve these 
complex and overlapping processes in the frontal lobe 
and elsewhere in the brain. In fact, just participating 
in the study engages the subject - and their frontal 
lobes - in complicated ways (for a good discussion of 
these issues, see Fuster, 2008). 

In order to disentangle the many processes engaged 
in any task, investigators studying frontal lobe function 


need to carefully design their studies so that they can 
identify which processes are specifically related to, for 
example, attention versus working memory, which are 
aspects of almost any task. In the following section, 
we present a brief summary of many neuroimaging 
studies that have looked at these prefrontal processes. 
As we stated in Chapter 4, a key approach to under¬ 
standing complex brain function is to have converging 
evidence across labs and studies, where many differing 
tasks and subject groups provide evidence in support 
of a central notion or hypothesis about brain regions 
and networks that subserve executive processes. 


6* 1 Attention and perception 

Imagine that you are a subject in an fMRI study. Your 
task is to look at a screen and when you see a picture 
of a male face, you are to press one button, and if you 
see a picture of a female face, your press another but¬ 
ton. Easy, right? What parts of the brain will be acti¬ 
vated by this task? You might suggest that visual 
cortex will be busy, based on your knowledge of sen¬ 
sory activity in the brain. You might also suggest that 
the fusiform face area (see Chapter 6) will Tight up' 
for this task. And you would be correct. But, there are 
other aspects of this task that will activate the brain: 
paying attention to the screen, making a decision 
about whether the face is male or female, preparing to 
press a button, and then actually pressing the button - 
all key aspects of frontal lobe function. In fact, this is 
a central finding in neuroimaging studies of visual or 
auditory perception - many areas in the brain outside 
of sensory cortex are activated. 

But first, a word about attention. In this chapter 
about frontal lobe functions, we will be discussing vol¬ 
untary attention : this is the aspect of attention in which 
we are in control of what we decide to pay attention 
to. When reading this sentence, for example, you are 
deciding to focus on the words and their meaning and 
not on the font type or size, or the sound of the clock 
ticking at your elbow. Voluntary attention often is called 
executive attention or top-down attention to indicate that it 
is the class of attentional processes that are under our 
control. Another type of attention is involuntary - these 
are automatic, frequently stimulus-induced processes 
whereby our attention is 'grabbed' by something in our 
environment. For example, the ringing of your cell, the 
sudden bright light when the sun comes from behind 
a cloud, the aroma of coffee brewing all may tempo¬ 
rarily attract your attention without any conscious 
effort on your part. Automatic attentional processes 
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can serve to disrupt our voluntary attention, but both 
are critical for being able to plan and initiate behavior - 
whether that behavior is finishing an essay for class 
in the morning or jumping out of the way when a car 
horn sounds behind you. The push-and-pull of vol¬ 
untary and involuntary attentional processes are key 
functions of the frontal lobes. 

An early investigation used PET to show localization 
patterns for sustained voluntary attention and provided 
important new information about prefrontal regions 
that subserved voluntary attentional processes (Pardo, 
Fox, & Raichle, 1991). At about the same time, seminal 
work on the voluntary attentional networks involved 
in perceptual tasks was provided by Michael Posner 
and Steve Peterson (Posner & Peterson, 1990), where 
they detailed an anterior attentional system, which they 
described as having three major separable networks 
that perform the alerting, orienting, and executive (con¬ 
flict resolution) voluntary attentional functions that are 
important to many tasks. Over the years, Posner and his 
colleagues developed an attention network test (ANT) 
that provided them with ways to separately measure 
and record brain regions that subserve these voluntary 
attentional networks. The anterior attentional system 
they describe has three key networks: alerting for main¬ 
taining an alert state, such as being ready to look at vis¬ 
ual images in the scanner example above; orienting such 
as preparing to see a new visual image of a face appears 
in the scanner, and executive , such as deciding whether 
the face is male or female. 

These networks are shown visually in the brain in 
Figures 12.8 through 12.10, where Figure 12.8 details 
fronto-parietal regions that are part of the alerting net¬ 
work, Figure 12.9 shows parietal lobe regions active in 
the orienting network, and Figure 12.10 shows frontal 
and many other regions active in the executive or con¬ 
flict resolution network. 

Another central finding from these and other studies 
of anterior voluntary attention networks is that the level 
of activity in the PFC corresponds to the level of attention 
required by the task, with more activity for tasks with 
higher attentional demands (Posner etal., 1988; Pardo 
et al., 1990). A part of the prefrontal region that typically 
is activated by tasks requiring focused attention is the 
anterior cingulate gyrus in the anterior cingulate cortex 
(ACC) (Posner et al., 1988; Raichle, 1994). 


6*2 Working memory 

The ability to keep something in mind for a limited 
amount of time is a central function in cognition. This 


Alerting network 


FIGURE 12.8 fMRI results for the alerting attentional net¬ 
work. The cross-section view of activations shows the thalamic acti¬ 
vations of the alerting effect. Source : Fan et al., 2005. 



Orienting network 


FIGURE 12.9 fMRI results for the orienting attentional net¬ 
work. The cross-section view of activations shows parietal activa¬ 
tion. Source: Fan et al., 2005. 


FIGURE 12.10 fMRI results for the conflict attentional net¬ 
work. The cross-section view of activations shows ACC activation. 
Source: Fan et al., 2005. 

ability - working memory - is closely associated with 
voluntary attentional systems. In fact, one way of 
describing working memory is that it serves as an 
inward directed voluntary attention system, directing 
attention to internal representations (Fuster, 2008). 
A central issue in investigating brain areas that sub¬ 
serve working memory is: how do you study working 
memory and separate findings from other executive 
processes such as attention and decision-making? To 
accomplish this, D'Esposito and colleagues devel¬ 
oped a dual task paradigm using two tasks that. 
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FIGURE 12.11 Outline of left cortex used in subsequent fig¬ 
ures to mark areas activated in working memory. Areas in convexity 
cortex designated with white labels; those in mesial cortex with gray 
labels. (SMA, supplementary motor area.) Below, temporal display 
of a trial in a typical visual working-memory (WM) task, delayed 
matching-to-sample, with faces. First upward inflexion of timeline 
marks the time of presentation of the sample face; second inflexion 
that of the choice faces. Delay-memory-period, between sample and 
choice, lasts 20 seconds. Source : Fuster, 2008. 


individually, did not have working memory demands 
(D'Esposito etal., 1995). Together, however, they did 
produce working memory demands and in this way, 
D'Esposito and colleagues were able to isolate pro¬ 
cesses that were specific to working memory function 
and not to general task performance. They identified 
regions in the PFC that were specifically involved in 
working memory processes, providing important new 
data on separable aspects of central executive func¬ 
tions in PFC. 

In his book. The Prefrontal Cortex , Fuster (2008) 
presents a meta-analysis of several neuroimaging 
studies of working memory to provide a schematic 
summary of brain areas involved in studies tapping 
visual versus verbal working memory. Results are pre¬ 
sented in Figures 12.11 through 12.13. 

Figure 12.11 shows a schematic of brain areas - both 
visual from the lateral view of the brain and those 
tucked inside - active in experiments that tap work¬ 
ing memory processes. Figure 12.12 shows activation 
patterns for a visual memory task, with activity in the 
occipital lobe as expected for an experiment using vis¬ 
ual stimuli, and frontal lobe regions that are active due 
to the nature of the task. Figure 12.13 shows a similar 
pattern of activation, but this time it is in response to a 
verbal memory task and so in this case, temporal lobe 
areas are active due to the auditory stimuli, along with 
frontal lobe regions. 



FIGURE 12.12 Relative (above-baseline) cortical activation at 
six moments in time (marked by yellow triangle) in the course of the 
visual memory task in Figure 12.11. Activations of convexity cortex 
in red, of medial cortex in pink. (1) At the sample, the activation is 
restricted to visual and posterior inferotemporal cortex; (2) in early 
delay, it extends to lateral prefrontal cortex, anterior cingulate, ante¬ 
rior inferotemporal cortex, and fusiform cortex; (3) in mid-delay, it 
persists in prefrontal, inferotemporal, and fusiform cortex; (4) in late 
delay, it migrates to premotor areas, persisting in inferotemporal 
and fusiform cortex; (5) at the response (choice of sample-matching 
face), it covers visual, inferotemporal, and fusiform cortex in the 
back, and extends to motor areas (including frontal eye fields), sup¬ 
plementary motor area (SMA), and orbitofrontal cortex in the front; 
(6) after the trial, activation lingers in anterior cingulate and orbito¬ 
frontal cortex. Source : Fuster, 2008. 


One hypothesis that has been put forth about the 
role of working memory systems/networks in the 
PFC is that their function may be to select the mate¬ 
rial or information required for the task at hand, 
whereas areas in the posterior, sensory areas of the 
brain are involved in the actual active maintenance of 
that material or information while it is being used 
in a given task (see Curtis & D'Esposito (2003) and 
Wager & Smith (2003) for reviews). The concur¬ 
rent activation of PFC and sensory cortices during 
executive cognitive functions has been described as 
'reverberating re-entry' by Fuster, with the networks 
involved in a perception-action cycle (Figure 12.14) 
(Fuster, 2008). 
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FIGURE 12.13 Cortical activation at six moments in time 
(yellow triangle) in the course of a verbal memory task: The memo¬ 
randum, in 1, is a word through earphones. Activations of convexity 
cortex in orange, of medial or sulcal cortex in yellow. (1) At the cue- 
memorandum, the activation is restricted to auditory cortex, supe¬ 
rior temporal gyrus, and inferior frontal cortex; (2) in early delay, 
it extends to lateral prefrontal, anterior cingulate, and superior- 
temporal and parietal association cortex; (3) in mid-delay, it persists 
in prefrontal and temporo-parietal cortex; (4) in late delay, it persists 
in prefrontal and migrates to premotor areas, while persisting in 
temporo-parietal cortex; (5) at the response (signaling whether cue 
word is on the screen), it covers visual and temporo-parietal cortex 
in the back, and extends to frontal eye fields, supplementary motor 
area (SMA), inferior frontal and orbitofrontal cortex in the front; (6) 
after the trial, activation lingers in anterior cingulate, orbitofrontal 
cortex, and language areas. Source : Fuster, 2008. 


6.3 Executive function and motor control 

We have described a central function of the PFC as 
the ability to plan our actions - whether mental or 
physical - and then to follow out that plan. The mental 
planning of motor action - from initial abstract repre¬ 
sentations to the actual motor codes - takes place in the 
frontal lobes. A current view of the neural organization 
for these processes is that the more abstract represen¬ 
tations/planning activities occur in the anterior por¬ 
tions of PFC, moving more posterior (and thus closer 
to the motor regions) as the activities become less 
abstract and move toward motor codes for movement 



FIGURE 12.14 Schematic diagrams of a monkey's brain 
showing cortical areas of reciprocal interaction during working 
memory at the top of the perception-action cycle. The prefrontal 
cortex interacts by reverberation with a different posterior corti¬ 
cal area depending on the modality of the memorandum (sample 
or cue) in the four working memory tasks depicted under the dia¬ 
grams. Source : Fuster, 2008. 


(for a review, see Fuster, 2008). The level of brain 
activity for planning and executing complex behav¬ 
iors corresponds to the level of difficulty of the action. 
A task frequently used to test frontal lobe executive 
function is the Tower of London task (Figure 12.15; 
Shallice, 1982). This task is like a puzzle with many 
steps for successful completion. In order to solve it, 
the subject needs to develop a plan. Researchers have 
found activation in DLPFC in the left hemisphere 
when solving this task, with higher levels of DLPFC 
activation found for subjects who found the task chal¬ 
lenging (Morris et ah, 1993). 

However, learning and practice change this effect, 
with well-learned - though complex - behaviors pro¬ 
ducing lower levels of brain activity (Figure 12.16) 
(Poldrack et ah, 2005; Posner & Raichle, 1994). Highly 
automatic behaviors like tying your shoe or locking 
a door produce little PFC activation. Thus, the feeling 
you may have that these behaviors are easy, requiring 
little effort on your part, corresponds to brain studies 
showing little PFC activation for them. 

We have briefly discussed the anterior cingulated 
cortex (ACC) (Brodmann area 24 in Figure 12.3). What 
is the role of the ACC in executive function? Although 
the role of the ACC in executive functions is still being 
elucidated, one hypothesis is that it has an inhibi¬ 
tory effect on frontal lobe processes. If this is the case, 
then the ACC may represent a functional part of the 
orbito-medial PFC that helps in reducing the effects of 
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FIGURE 12.15 The Tower of London test. Source : Fuster, 2008. 


Naive Practiced Novel 



FIGURE 12.16 PET activation during performance of a 
verb-generation task. The subject is given a series of nouns by the 
investigator and required to produce a verb appropriate to each. 
The control task was simply the reading aloud of the names as they 
appeared on a TV monitor. Subtractive scans of three brain slices in 
three different conditions (left to right): performance by naive sub¬ 
ject, subject practiced with the same list of nouns, and subject pre¬ 
sented with a new list of nouns. Source : Fuster, 2008, from Posner 
and Raichle (1994), with permission. 


distracting influences on the executive planning func¬ 
tion (Figure 12.17; Bush etal., 2000). Fuster (2008) 
hypothesizes that this inhibitory control that resists 
distracting influences may serve as the flip side of 
executive attentional processes. Thus, in this way your 
success at focusing on a task at hand is aided both by 
your ability to pay attention to it and by your ability 
to ignore distractions. 


6.4 Decision-making 

In our everyday life, we encounter two types of situa¬ 
tions that require two different ways to address them. 
Situations like balancing a checkbook and remember¬ 
ing a phone number or a name are deterministic. Each 
of them has a single correct solution intrinsic in the 
situation, all the other responses being false. By find¬ 
ing the correct solution, we engage in veridical deci¬ 
sion-making. Situations like deciding what to wear, 
which movie to see, or which career path to choose are 
ambiguous. They do not have an intrinsically correct 
solution. By making our choice, we engage in adaptive 
(or actor-centered) decision-making. 

In school, we are given a problem and must find 
the correct answer. Only one correct answer usually 
exists. But aside from high school exams, college tests, 
and factual and computational trivia, most decisions 
we make in our everyday lives do not have intrinsi¬ 
cally correct solutions. The choices we make are not 
inherent in the situations at hand. They are a com¬ 
plex interplay between the properties of the situations 
and our own properties, our aspirations, our doubts, and 
our histories. The prefrontal cortex is central to such 
decision-making. Finding solutions for determinis¬ 
tic situations often is accomplished algorithmically. It 
increasingly is delegated to various devices: calcula¬ 
tors, computers, directories of all kinds, but making 
choices in the absence of inherently correct solutions 
remains, at least for now, a uniquely human territory. 
In a sense, the freedom of choice is possible only when 
ambiguity is present. 

Resolving ambiguity often means choosing the 
question first, that is, reducing the situation to a ques¬ 
tion that does have a single correct answer. Precisely 
how we disambiguate the situation depends on our 
priorities at the moment, which themselves may 
change depending on the context. An inability to 
reduce ambiguity leads to vacillating, uncertain, 
inconsistent behavior. 

At the same time, an individual must have the flex¬ 
ibility to adopt different perspectives on the same 
situation at different times. The organism must be 
able to disambiguate the same situation in a multiple 
of different ways and to have the capacity to switch 
between them at will. Dealing with inherent ambiguity 
is among the foremost functions of the frontal lobes. 
Studies have shown that patients with frontal-lobe 
damage approach inherently ambiguous situations 
differently from the way healthy people do. The loss of 
the ability to make decisions is among the most com¬ 
mon signs of early dementia. Damage to other parts of 
the brain does not seem to affect these processes. 
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FIGURE 12.17 Confirming separate cognitive and emotional conflict regions. A summary of many studies of the 
anterior cingulate cortex, showing different cognitive and emotional regions. Source : Bush et ah, 2000. 


Where do the processes underlying decision-making 
occur in the frontal lobes? Converging evidence impli¬ 
cates the orbito-medial PFC, especially when there 
are emotional factors in the decision-making process. 
Think about playing a hand of poker. You must con¬ 
stantly make decisions about what to do next with 
incomplete information - should you fold, raise . . . 
what will your competitors do? In order to do well at 
this game, you need to assess the risk factors and the 
rewards. Bechara and colleagues have investigated 
these processes using the Iowa gambling task (Bechara, 
Damasio, Tranel, & Anderson, 1994). They and many 
other researchers investigating the neural bases for 
assessing risk and reward have shown that the orbital 
or medial PFC is activated during these tasks. 

Have you ever been in a state of internal conflict 
over a decision that you are making? With a real sense 
that 'part of you' wants you to listen to your heart and 
another 'part of you' wants you to listen to your head ? 
This type of internal conflict between emotional feel¬ 
ings and rational thoughts is proposed to reflect the 
trading relationship between the orbito-medial PFC, 
with its connections to subcortical emotional regions, 
and lateral PFC, with its connections to executive con¬ 
trol regions. 

6.5 Rule adoption 

In order to navigate our way through our complex 
daily lives, it is critical to develop ways to short-cut 


all the things that we need to plan for and carry out. 
Humans are wonderful rule adopters - we develop 
and learn strategies for streamlining our busy lives. 
Like a strategic plan or a schema, rules help us 
increase our efficiency. The Wisconsin Card Sorting 
test (shown in Figure 12.18) is a good example of the 



FIGURE 12.18 The Wisconsin Card Sorting Test (WCST). The 
cards can be sorted by matching the color of the item on the card, or 
the number of items, their shape, or even if they don't match on any of 
these features. At the beginning of the game, the experimenter deter¬ 
mines which matching 'rule' to be used (such as 'match the new card 
by color') but he does not tell the player the rule - the player must 
learn it by trial and error. During the game, the experimenter will 
change the rule and again, the player must learn the new rule through 
trial and error. This is a test of mental flexibility: the ability to learn a 
new rule and to adapt to a new rule when needed. Source : Fuster, 2008. 
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FIGURE 12.19 In this study, the experimenters wanted to see 
which brain areas were active while subjects implemented either sim¬ 
ple or compound abstract 'rules'. A rule cue was provided followed 
by a long (7-15 s) delay. Brain areas more active for compound versus 
simple cues are shown at the top of this figure. Areas active during 
the delay (while the subject is keeping the rule in mind) are shown in 
the middle (simple rule) and bottom (compound rule) of the figure. 
Source : Bunge, 2004, as adapted in Fuster, 2008. 

mental flexibility humans have in acquiring rules and, 
importantly, in changing them when needed. 

Neuroimaging studies of rule learning in PFC have 
shown that, in a manner similar to attentional and 
working memory demands, neural activity in the fron¬ 
tal regions increases with the complexity of the rule set 
to be learned or carried out (Figure 12.19; Bunge, 2004). 

Neuroimaging studies have shed new light on the 
many and diverse operations carried out - or directed - 
by the PFC, from paying attention to a stimulus in 
your environment, to monitoring how it is changing, 
to keeping something in mind, to complex decision¬ 
making. Many of these processes are highly over¬ 
lapping in time and neural regions, thus we are still 
elucidating which frontal lobe areas contribute to these 
processes. Although we are still in the early stages 
of understanding just how and where executive pro¬ 
cesses are being done in the PFC, converging evidence 


from neuroimaging studies are beginning to present a 
clearer picture of PFC function. 

7*0 FRONTAL LOBE DYSFUNCTION 


We have discussed many functions of the frontal lobe 
including voluntary attention, working memory, deci¬ 
sion making, and even your personality. What hap¬ 
pens when this critical area of the brain is damaged? 
Or fails to develop in a typical manner? The answers 
to these questions are as complex as the frontal lobes 
themselves. 

7.1 The fragile frontal lobes 

Frontal lobe dysfunction often reflects more than direct 
damage to the frontal lobes (Goldberg, 1992). The fron¬ 
tal lobes seem to be the bottleneck, the point of con¬ 
vergence of the effects of damage virtually anywhere 
in the brain. There is a reciprocal relationship between 
frontal and other brain injuries. Damage to the frontal 
lobes produces wide ripple effects through the whole 
brain. At the same time, damage anywhere in the brain 
sets off ripple effects interfering with frontal lobe func¬ 
tion. This unique feature of the frontal lobes reflects 
its role as the Traffic hub' of the nervous system, with 
a singularly rich set of connections to and from other 
brain structures. This makes frontal lobe dysfunction 
the most common and least specific finding among 
neurological, psychiatric, and neurodevelopmental 
conditions (Goldberg, 1992). 

The frontal lobes' exceptionally low 'functional 
breakdown threshold' is consistent with Hughlings 
Jackson's concept of 'evolution and dissolution' (1884). 
According to Jackson's proposal, the phylogenetically 
(evolutionary) youngest brain structures are the first 
to succumb to brain disease. The frontal lobes' unique 
vulnerability is probably the flip side of the excep¬ 
tional richness of their connections. A frontal lobe dys¬ 
function does not always signify a direct frontal lobe 
lesion. Instead, it may be a remote effect of a diffuse, 
distributed, or distant lesion. 

7.2 Frontal lobe syndromes 

The importance of executive functions can be best 
appreciated through the analysis of their disintegra¬ 
tion following brain damage. A patient with dam¬ 
aged frontal lobes retains, at least to a certain degree, 
the ability to exercise most cognitive skills in isolation 
(Luria, 1966). Basic abilities such as reading, writing. 
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FIGURE 12.20 The orbitofrontal region is just above the orbits of the eyes. The orbitofrontal cortex (green stripes) can 
be distinguished in the ventral prefrontal lobe. Orbitofrontal cortex is involved in understanding future rewards, changes in 
reward contingencies, and goal selection. Damage to the orbitofrontal region may lead to a loss of behavioral inhibition. Source : 
Davidson and Irwin, 1999. 


simple computations, verbal expression, and move¬ 
ments may remain largely unimpaired. Deceptively, 
the patient will perform well on the psychological 
tests measuring these functions in isolation. However, 
any synthetic activity requiring the coordination of 
many cognitive skills into a coherent, goal-directed 
process will become severely impaired. 

Damage to different parts of the frontal lobes 
produces distinct, clinically different syndromes. 
The most common among them are the dorsolateral 
and orbitofrontal prefrontal syndromes (Figure 12.20; 
Goldberg and Costa, 1985). 

7.2.1 Dorsolateral prefrontal syndromes 

Most common symptoms of dorsolateral prefron¬ 
tal syndromes are personality changes , field-dependent 
behavior , and perseverative behavior. 

Personality changes 

Clinically, a patient with dorsolateral frontal syn¬ 
drome will be characterized by impaired ability to ini¬ 
tiate behaviors. He or she tends to show 'flat affect'; 
that is, monotonous speech and sense of indifference. 
The patient is neither sad nor happy; in a sense, he or 
she has no mood. This state of indifference persists 
no matter what happens to the patient, good things 
or bad. However, the most conspicuous feature of a 
patient with dorsolateral syndrome is a drastically 
impaired ability to initiate behaviors. Once started in 


a behavior, however, the patient is equally unable to 
terminate or change it on his or her own. Such com¬ 
bined 'inertia of initiation and termination' is seen in 
various disorders affecting the frontal lobes, including 
chronic schizophrenia. 

Following even relatively mild head trauma, it 
is common for patients to become indifferent and 
devoid of initiative and drive. The change may be 
subtle, and it is not always apparent to family mem¬ 
bers or even physicians that the change is neurologi¬ 
cal in nature, that it is a mild form of the frontal lobe 
syndrome. These symptoms are often called person¬ 
ality change, but personality is determined to a large 
extent by neurobiology. The frontal lobes have more to 
do with our personalities than any other part of the 
brain, and frontal lobe damage can produce profound 
personality change. 

Field-dependent behavior 

Another common symptom of dorsolateral prefrontal 
syndromes is that the patient is at the mercy of inci¬ 
dental distractions and thus is unable to follow inter¬ 
nally generated plans. In extreme cases it may take 
the form of field-dependent behavior. A frontal lobe 
patient will drink from an empty cup, put on a jacket 
belonging to someone else, or scribble with a pencil on 
the table surface, merely because the cup, the jacket, 
and the pencil are there, even though these actions 
make no sense. This phenomenon was studied exten¬ 
sively by the French neurologist Francois Lhermitte, 
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who called it utilization behavior (Lhermitte, 1983). 
In the most extreme cases, field-dependent behavior 
takes the form of direct imitation, called echolalia (imi¬ 
tation of speech) or echopraxia (imitation of action). 

Such patients will perform particularly poorly 
(Stuss et ah, 2001) on the Stroop Test (Stroop, 1935). 
What makes the Stroop Test so interesting? It requires 
that we go against our immediate impulse. The 
impulse is to read the words; this is the natural ten¬ 
dency of every literate person when you see written 
material. But the task is to name the colors. To com¬ 
plete the task successfully, we must follow the internal 
plan, the task, against our natural, entrenched tendency 
(see also Chapters 2 and 8). 

When neurological illness affects the frontal lobes, 
the ability to stay on track often becomes lost, and the 
patient is often at the mercy of incidental environmen¬ 
tal stimuli and tangential internal associations. Easy 
distractibility is a feature of many neurological and 
psychiatric disorders, and it is usually associated with 
frontal lobe dysfunction. For example, attention deficit 
hyperactivity disorder (ADHD), with its extreme dis¬ 
tractibility, is usually linked to frontal lobe dysfunction 
(Barkley, 1997). The relationship between the frontal 
lobes' role in guiding behavior by internal representa¬ 
tion during performance on Stroop Test, ADHD, and 
emotional valence of a task was elucidated in a com¬ 
bined analysis of multiple studies (Figure 12.17). 

Mental rigidity and perseveration 
Our ability to maintain mental stability has to be bal¬ 
anced by mental flexibility. No matter how focused we 
are on an activity or a thought, there comes a time 
when the situation calls for doing something else. 
Being able to change one's mindset is as important 
as staying mentally on track. The capacity to switch 
with ease from one activity or idea to another is so 
natural and automatic, that we take it for granted. 
In fact, it requires complex neural machinery, which 
also depends on the frontal lobes. Mental flexibil¬ 
ity, the ability to see things in a new light, creativity, 
and originality all depend on the frontal lobes. More 
profound forms of mental rigidity produce obsessive- 
compulsive disorder (OCD), in which dysfunction of the 
caudate nuclei closely linked to the frontal lobes has 
been implicated (Rauch et ah, 1994). 

Frontal lobe damage often produces extreme 
mental rigidity, which may severely undermine the 
patient's cognition. Quite often a closer look at a fron¬ 
tal patient's performance on a number of tasks shows 
that complete transition from one task to another is 


impossible, and fragments of a previous task attach 
themselves to the new one. This phenomenon is called 

perseveration. 

A seemingly simple neuropsychological test is 
quite sensitive to subtle impairment of mental flex¬ 
ibility. The test, known as the Wisconsin Card Sorting 
Test (Grant and Berg, 1948), requires the subject to sort 
cards with simple geometric forms into three catego¬ 
ries according to a simple principle (Figure 12.18). The 
classification principle is not revealed in advance and 
the subject must establish it through trial and error. 
But when the principle finally has been mastered it 
abruptly changes unbeknownst to the subject. Once 
the subject catches up with the new principle, the 
principle is changed without forewarning again, and 
again, and again. The task requires planning, guid¬ 
ance by internal representation, mental flexibility, and 
working memory - in short all the aspects of frontal 
lobe function that we have discussed. 

7.2.2 Orbitofrontal prefrontal syndromes 
and self-control 

The orbitofrontal prefrontal syndrome is, in many ways, 
the opposite of the dorsolateral syndrome. The patients 
are behavior ally and emotionally disinhibited. Their 
affect is rarely neutral, constantly oscillating between 
euphoria and rage, with impulse control ranging from 
poor to nonexistent. Their ability to inhibit the urge 
for instant gratification is severely impaired. They do 
what they feel like doing, when they feel like doing it, 
without any concern for social taboos and legal pro¬ 
hibitions. They have no foresight of the consequences 
of their actions. A patient afflicted with the orbitofron¬ 
tal syndrome (due to head injury, cerebrovascular ill¬ 
ness, or dementia) may engage in shoplifting, sexually 
aggressive behavior, reckless driving or other actions 
commonly perceived as antisocial. These patients are 
known to be selfish, boastful, puerile, profane, and 
sexually explicit. Their humor is off-color and their 
jocularity, known as Witzelsucht, resembles that of a 
drunken adolescent (Oppenheim, 1889). If the dorso¬ 
lateral patients are in a sense devoid of personality, 
then orbitofrontal patients are conspicuous for their 
'immature' personality. 

7.2.3 Reticulofrontal disconnection syndrome 

In cases when the frontal lobes themselves are struc¬ 
turally intact but the patient presents with frontal lobe 
symptoms, the problem may lie with the pathways 
connecting frontal lobes to some other structures. 
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Damage to these pathways may result in a condition 
known as the reticulofrontal disconnection syndrome 
(Goldberg et al., 1989). 

The brainstem contains the nuclei thought to be 
responsible for the arousal and activation of the rest of 
the brain. A complex relationship exists between the 
frontal lobes and the brainstem reticular nuclei, which 
are in charge of activation and arousal. The relation¬ 
ship is best described as a loop. On the one hand, the 
arousal of the frontal lobes depends on the ascending 
pathways. On the other hand, there are pathways pro¬ 
jecting from the frontal lobes to the reticular nuclei of 
the ventral brainstem. Through these pathways the 
frontal lobes exert their control over the diverse brain 
structures by modulating their arousal level. If the 
frontal lobes are the decision-making device, then the 
brainstem structures in question are an amplifier help¬ 
ing communicate these decisions to the rest of the brain 
in a loud and clear voice. The descending pathways are 
the cables through which the instructions flow from 
the frontal lobes to the critical ventral brainstem nuclei. 

We can easily see how damage to the pathways 
between the brainstem and the frontal lobes may dis¬ 
able executive functions without actually damaging 
the frontal lobes per se. 


7.3 Frontal lobe damage and asocial behavior 

The relationship between frontal lobe damage and 
asocial behavior is particularly intriguing and com¬ 
plex. It has been suggested, based on several pub¬ 
lished studies, that the prevalence of head injury is 
much higher among criminals than in the general pop¬ 
ulation, and in violent criminals than in nonviolent 
criminals (Volavka et al., 1995; Raine et al., 1997). For 
reasons of brain and skull anatomy, closed head injury 
is particularly likely to affect the frontal lobes directly, 
especially the orbitofrontal cortex. Furthermore, dam¬ 
age to the upper brainstem is extremely common in 
closed head injury, even in seemingly mild cases, and 
it is likely to produce frontal lobe dysfunction even 
in the absence of direct damage to the frontal lobes 
by producing the 'reticulofrontal disconnection syn¬ 
drome' (Goldberg et al., 1989). 

Adrian Raine and his colleagues (Raine et al., 1997) 
studied the brains of convicted murderers with PET 
scans and found abnormalities in the PFC. Raine and 
colleagues (2000) also studied the brains of men with 
antisocial personality disorder and found an 11% 
reduction in the gray matter of their frontal lobes. The 
cause of this reduction is uncertain, but Raine believes 


that this reduction is at least in part congenital, as 
opposed to caused by environmental factors such as 
abuse or bad parenting. The link between frontal lobe 
dysfunction and asocial behavior raises an important 
set of social, moral, and legal issues far beyond the 
scope of this chapter. 

7.4 Other clinical conditions associated 
with frontal lobe damage 

PFC is afflicted in a wide range of conditions 
(Goldberg, 1992; Goldberg and Bougakov, 2000) and it 
is not necessary to have a focal frontal lesion to have 
prefrontal dysfunction. The frontal lobes are particu¬ 
larly vulnerable in numerous nonfocal conditions. 
Such disorders as schizophrenia (Ingvar and Franzen, 
1974; Franzen and Ingvar, 1975), traumatic brain 
injury (TBI) (Deutsch and Eisenberg, 1987), Tourette's 
syndrome (Tourette, 1885; Shapiro and Shapiro, 
1974; Sacks, 1992), and attention deficit (hyperactiv¬ 
ity) disorder (AD(H)D) (Barkley, 1997) are known to 
involve frontal lobe dysfunction. In TBI, for instance, 
the frontal lobe dysfunction can be caused by either 
direct frontal injury, or by injury disrupting the 
brainstem-frontal connection (Goldberg et al., 1989). 
Executive functions are also compromised in demen¬ 
tia and in depression. 

It appears that it is not necessary to have a frontal 
lobe lesion to have frontal lobe syndrome. There are 
many conditions where, based on functional neu¬ 
roimaging and neuropsychological studies, frontal 
lobe dysfunction is present, but there is no evidence 
of structural morphological damage to the frontal 
lobes. 

Attention deficit (hyperactivity) disorder 
(AD(H)D) 

The prefrontal cortex and its connections to the ven¬ 
tral brainstem play a particularly important role in 
the mechanisms of attention. When we talk about the 
attention deficit (hyperactivity) disorder (AD(H)D), 
we usually implicate these systems. The exact causes 
of damage to these systems vary. They may be inher¬ 
ited or acquired early in life. They may be biochemi¬ 
cal or structural. As we already know, frontal lobes 
are particularly vulnerable in a very broad range of 
disorders, hence the very high rate of frontal lobe dys¬ 
function. The way the diagnosis of AD(H)D is com¬ 
monly made, it refers to any condition characterized 
by mild dysfunction of the frontal lobes and related 
pathways in the absence of any other, comparably 
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severe dysfunction. Given the high rate of frontal lobe 
dysfunction due to a variety of causes, the prevalence of 
genuine AD(H)D should be expected to be very high. 

To understand better the proposed mechanism of 
AD(H)D we need to understand the nature of another 
aspect of executive control - selective attention. The 
goal of action must be identified and it must effec¬ 
tively guide behavior for a period of time. We already 
know that goal-setting and goal-maintenance are 
provided by the prefrontal cortex. The prefrontal cor¬ 
tex exerts its influence on the posterior aspects of the 
cortical hemispheres. These are the structures most 
directly involved in processing the incoming infor¬ 
mation. Depending on the goal at hand, distinct, par¬ 
ticular parts of the posterior cortex must be brought 
into the state of optimal activation. The selection of 
these areas is accomplished by the prefrontal cortex. 
The prefrontal cortex exerts its influence through the 
nuclei of the ventral brainstem, which can selectively 
activate vast cortical regions through their ascending 
projections. The prefrontal cortex guides the influence 
of these nuclei on the posterior cortices through its 
own descending pathways into the ventral brainstem. 
Finally, the prefrontal cortex modifies its control over 
brainstem nuclei, based on the feedback it receives from 
the posterior cortex. 

In sum, attention can best be described as a loop¬ 
like process involving complex interactions between 
the prefrontal cortex , ventral brainstem (and possibly 
also nonspecific midline thalamic nuclei ), and posterior 
cortex (Figure 12.21). Breakdown anywhere along this 
loop may interfere with attention, thus producing a 
form of attention deficit disorder. Therefore, any dam¬ 
age to the prefrontal cortex or its pathways may result 
in attentional impairment. 

8*0 A CURRENT VIEW OF ORGANIZING 
PRINCIPLES OF THE FRONTAL LOBES 


After decades of research into frontal lobe function 
in nonhuman primate and in human, using many 
experimental techniques and methods, some organiz¬ 
ing principles have emerged. A leader in the field of 
frontal lobe research is Joaquin Fuster, who proposed 
a model for frontal lobes function. He states: 

1 The entirety of the cortex of the frontal lobe is 
devoted to the representation and production of 
action at all levels of biological complexity 

2 The neuronal substrate for the production of 
any action is identical to the substrate for its 
representation 



FIGURE 12.21 An attentional loop combining frontal, brain¬ 
stem, and posterior cortex. Goldberg (2001) proposed that atten¬ 
tional functions may be influenced not just by the frontal lobes, but 
by a causal loop extending downward into the reticular formation 
of the brainstem, going upward to posterior cortex. This view is 
consistent with evidence regarding the brainstem arousal system. 
Source : Elkhonon Goldberg, with permission. 


3 That substrate is organized hierarchically, with the 
most elementary actions at low levels of the hier¬ 
archy, in orbitofrontal and motor cortex, and the 
most complex and abstract actions in lateral pre¬ 
frontal cortex 

4 Frontal-lobe functions are also organized hierar¬ 
chically, with simpler functions nested within, and 
serving, more global functions. (Fuster, 2008, page 
334.) 

The notion of a hierarchical organization for simple 
versus complex, abstract versus concrete frontal lobe 
function has been developed in neuroimaging stud¬ 
ies of human frontal lobe processes. A recent review 
article by Badre (2008) provided a summary of find¬ 
ings to date, including models developed by Koechlin, 
D'Esposito, and others, showing data supporting the 
view of a hierarchal organization of the frontal lobe, 
with an anterior-to-posterior organization that is 
based on the degree of abstraction (Figure 12.22). 


Neuroimaging Summary 

While many more details clearly need to be discovered 
before we have a unitary explanation of frontal lobe 
function, recent advances in neuroimaging techniques 
and methods have helped to increase our knowledge 
of the complex role the frontal lobes, and in particu¬ 
lar the PFC, play in human cognition. However, even 
though neuroimaging studies may serve to guide us as 
to specific regions where certain executive processes - 
such as voluntary attention or working memory - 
may be located, they show a dramatically different 
picture of human executive function than is seen when 
observing a patient with frontal lobe damage. How 
do results from neuroimaging studies correspond to 
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FIGURE 12.22 Results from the response, feature, dimension, 
and context experiments and comparison to the model of Koechlin 
et al. (2003). Abstract relational hierarchy seems to provide a par¬ 
simonious account of rostro-caudal gradient across the models of 
Koechlin et al. (2003) and Badre and D'Esposito (2007). Spheres with 
diameters of 8 mm (within the smoothing kernel of each experiment) 
were centered on maxima from response (i), feature (iii), dimension 
(v), and context (vi) manipulations of Badre and D'Esposito (2007) 
(red), and on the sensory (ii), context (iv), and episodic (vi) manipu¬ 
lations of Koechlin et al. (2003) (blue). These spheres were rendered 
on an inflated Talairach surface. Note that the spheres are for pre¬ 
cise illustration of proximity but do not represent actual spread of 
activation in each experiment. Broken lines separate manipulations 
at equivalent levels of abstraction in a representational hierarchy. 
Equivalent episodic and contextual control manipulations across 
the two experiments are also labeled. Source : Badre, 2008, adapted 
from Badre and D'Esposito, 2007, with permission. 


studies of patients with frontal lobe syndromes? These 
two sets of findings are not easy to bring together into 
a cohesive whole at present. Neuroimaging studies 
have provided a wealth of data about specific regions 
and locations that activate for various aspects of fron¬ 
tal lobe processes but the results don't correspond 
directly to what we know about human behavior 
when frontal lobes are damaged. This is the challenge 
for cognitive neuroscientists - to continue to provide 
converging evidence across many techniques and sub¬ 
ject groups in order to elucidate the underlying organ¬ 
izational principles for the complex and uniquely 
human PFC, the 'organ of civilization'. 


9.0 TOWARD A UNIFIED THEORY 
OF EXECUTIVE CONTROL: A 
CONCLUSION 


To summarize, after having been overlooked for many 
decades, executive functions have become the focus of 
an ever-increasing body of research. 


Early neuroimaging studies of frontal lobe functions 
produced a complex patchwork of regions specialized 
for sensory modalities, linguistic versus nonlinguistic 
processes, object-oriented versus spatially-oriented, and 
'what' versus 'where' pathways. The early approach of 
elucidating capsulated 'modules' for the PFC produced 
many important findings but ultimately did not pro¬ 
vide a clear picture of the organizational principles for 
the PFC, nor did they correspond well with the wealth 
of information about frontal lobe syndromes. 

A very different approach traces its lineage to 
the work by the great Russian neuropsychologist 
Alexander Luria (1966). A continuation and extension 
of Luria's original theory can be seen in the relatively 
recent trend toward the refutation of the modular 
view of functional neocortical organization in favor of 
the distributed-emergent principle of functional corti¬ 
cal organization (Goldberg et al., 1989; Goldberg, 1992; 
Fuster, 2003). According to Goldberg's gradiental the¬ 
ory, the functional organization of heteromodal associ¬ 
ation cortices (such as prefrontal cortices) is interactive 
and distributed. The heteromodal association cortex 
develops along the continuous distributions. In these 
distributions (called gradients) functionally close 
aspects of cognition are represented in anatomically 
close areas of the association neocortex. 

Yet another theory of cortical representation elabo¬ 
rating on Luria's functional systems theory was put 
forth by Joaquin Fuster (2003). Fuster maintains that 
cognitive functions do not have discrete cortical repre¬ 
sentation. In his theory, he introduces a reentrant unit, 
called cognit , which he proposes as a generic term for 
any representation of knowledge in the cerebral cortex. 
Cognits are dynamic structures which, in neural terms, 
roughly coincide with neuronal assemblies and the 
connections between the neurons. According to Fuster, 
cognitive functions are represented by information 
exchange within and between cognits, and different 
cognitive functions draw upon many overlapping cog¬ 
nits. The crucial tenet of Fuster's theory is that differ¬ 
ent cognits (neural networks) have identifiable cortical 
distribution but cognitive functions that use them do 
not, since different functions may rely on the same or 
similar circuits. 

According to these theories, the nature of corti¬ 
cal representation of executive control is distributed 
as well as localized. Executive control can be consid¬ 
ered unitary in the sense that it is in charge of actions, 
both external and internal, and works to integrate 
such factors as time, novelty, complexity, and possibly 
ambiguity. 
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10-0 DRAWING EXERCISES 
AND STUDY QUESTIONS 

1 What are some of the functions attributed to the 
prefrontal lobe? 

2 Discuss a current debate about the specificity of 
prefrontal functions. 

3 In the outline brain shown in Figure 12.23: 

a. Color the major lobes 

b. Label major landmarks, like the central sul¬ 
cus and Sylvian fissure. 

c. Label the major gyri (green lines) and sulci 
(dotted red lines). 





For Charles Darwin , it was obvious beyond any need for argument that non-human animals 
are sentient \ . .the lower animals , Zi/ce man’, he wrote in 1871, ‘manifestly feel pleasure and 
pain , happiness and misery ’ (p. 39). ‘The fact that the lower animals are excited by the same 
emotions as ourselves is so well established that it will not be necessary to weary the reader by 
many details’ (However,) ... in the belief that identifying problems and finding ways to answer 
them is a way to move forwards , I shall first emphasize why sentience is still a profound prob¬ 
lem , despite the ease with which Darwin spoke about the mental experiences of animals. 

Marian Stamp Dawkins (2006) 



The basic emotional brain is highly conserved among mammals, and possibly even more widely. Upper left, a human brain seen from the 
midline (medially), showing how information can flow from visual cortex (VI) to the amygdala, an emotional 'hub' for neuronal traffic 
from many different sources. Snakes appear to be a biological stimulus for fear for both humans and other mammals. On the opposite 
pole, cradling young birds and other newborns seems to evoke a soothing and calming effect (Panksepp, 1998). Obviously humans have 
a giant neocortex, which modifies basic mammalian emotional systems centered in the limbic (core) brain. 
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1.0 INTRODUCTION 

Do you remember where you were on the morning of 
September 11, 2001? You likely do recall that day, and 
you probably do not have a strong memory for, say, 
September 9 of that year. Why is this? The events of 
9/11 are firmly etched in our brains because of the 
highly charged emotions that are entwined with the 
memory of the occurrences of that day In previous 
chapters, we have discussed higher cognitive brain 
functions such as memory, executive functions, and 
language. Our emotions are strongly coupled with 
the brain systems underlying these cognitive func¬ 
tions. In this chapter, we will explore the emotion 

Cognition, Brain, and Consciousness, edited by B. J. Baars and N. M. Gage 
ISBN: 978-0-12-375070-9 


systems of the brain and their interactions with cogni¬ 
tive processing. The term 'affective neuroscience' was 
coined in the late 1990s to denote a new area of study: 
the neuroscience of emotion. While a great deal is now 
known about emotion in the brain, we will focus here 
on how emotion systems shape and influence cogni¬ 
tive processes. 

1.1 The triune brain 

Paul MacLean introduced the 'triune brain' concept 
in the 1960s to describe the functionally distinct lay¬ 
ers of the mammalian brain. It has become a widely 
used way of thinking about the overall functional 
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organization of the brain. While some aspects of the 
triune brain theory have been controversial, neverthe¬ 
less, it remains a helpful way to think about different 
'layers' of the mammalian brain. In MacLean's view, 
the brain developed over the course of vertebrate evo¬ 
lution into a three-layered organ, where these layers 
retain some of the separateness of their different evo¬ 
lutionary origins despite being highly interconnected. 
We can see the effects of each of the layers in human 
behavior, especially in the relationship of cognition 
and emotion. 

The oldest layer of the brain is called the reptilian 
brain. It is composed of the brainstem (medulla, pons, 
cerebellum, midbrain, globus pallidus, and olfactory 
bulbs) - the structures that dominate in the brains of 
snakes and lizards. This brain layer does not learn very 
well from experience but is inclined to repeat instinc¬ 
tual behaviors over and over in a fixed way. In humans, 
this part of the brain controls survival activities like 
breathing, heart rate, and balance. We will not have 
much to say about the reptilian brain in this chapter. 

The mammalian brain is layered over the reptilian 
brain (Figure 13.1). It consists primarily of a system of 
brain parts called the limbic system. The word 'limbic' 
comes from the Latin word limbus and means 'bor¬ 
der, edge, or hem' - it refers to the location inside the 
cerebral hemispheres around the edge of the lateral 



FIGURE 13.1 The triune brain: orange represents neocortex, 
green is the mammalian brain, and yellow is the reptilian brain. 


ventricles (fluid-filled spaces). The limbic system was 
first recognized in the late 1800s, but an understand¬ 
ing of its function in emotion did not develop until 
the work of neuroanatomist Papez was published in 
1937. Another older name for the limbic system is the 
Papez circuit. 

The list of component parts of the limbic system 
varies depending on the researcher that we consult - 
there seems to be no universal agreement about what 
the limbic system actually consists of! Some neuro¬ 
scientists think that we should no longer speak of a 
limbic system at all. We will retain the term as a use¬ 
ful organizing concept for a set of related subcortical 
brain parts that support our emotional life. Commonly 
cited constituents are the amygdala, hippocampus, 
parahippocampal cortex, cingulate gyrus, hypothala¬ 
mus, and ventral striatum/nucleus accumbens. 

The limbic system has a major role in human emo¬ 
tion. We see the effects of the limbic system in our 
conscious experience in the added valence (positive 
or negative value or feeling) and salience ('notice¬ 
ableness') of particular images and thoughts. We share 
this part of our brain with other mammals. Prototypic 
mammalian emotional responses are easily recogniz¬ 
able in our pet dogs and cats. 

In terms of the adaptive role of the limbic system, 
we can say that the limbic system contains several 
distinct systems evolved to respond to mammalian 
evolutionary pressures such as danger, reproductive 
and nurturance needs, and acquisition of food. For 
example, the amygdala and hypothalamus cooperate 
in an early warning system for danger, initiating sur¬ 
vival maneuvers automatically when confronted with 
stimuli similar to those encountered in past dangerous 
situations. However, the amygdala's stimulus process¬ 
ing has low resolution of details compared to that of 
the sensory areas of the cerebral cortices. Stimulus rec¬ 
ognition in the limbic system follows a 'close enough 
is good enough' rule, so we sometimes find ourselves 
jumping at the site of a long, dark coil in the grass, 
only to find that it is a garden hose and not a snake. 
Following the conditioning of evolutionary history, 
the limbic system would 'rather be safe than sorry'. 
In our conscious experience, we find ourselves acting 
without voluntary initiation under the influence of 
the amygdala. Moments later, our rational selves feel 
embarrassment at our apparently silly behavior. We 
can easily find other examples of the interplay of the 
limbic system and the cortical system in our everyday 
behavior. We will look more closely at this shortly. 

The limbic system is densely interconnected with 
the cortex, particularly through the orbital gyri of the 
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VM-PFC (ventromedial prefrontal cortex) and also 
the insular cortex. The dense connections with the 
VM-PFC have led to its nickname as the feeling part of 
the thinking brain. VM-PFC is a gateway between the 
limbic system and neocortex. 

The neocortex or primate brain is the most recent 
addition to our brains. It consists of the wrinkled 
covering of the cerebral hemispheres (as well as 
some subcortical nuclei, such as the basal ganglia), 
which has mushroomed in non-human primates and 
humans compared to other mammals. In humans, the 
neocortex is the home of our complex cognitive, lin¬ 
guistic, motor, sensory, and social abilities. The neo¬ 
cortex gives us considerable flexibility and creativity 
in adapting to a changeable environment. Cortex func¬ 
tions to socialize and control expression of emotions 
that originate in the limbic system; cortical appraisal 
of situations is also necessary for a more nuanced 
emotional repertoire than is possible based on the 
functioning of the limbic system alone. 


1.2 Basic emotions and the role of reflective 
consciousness 

Emotional responsiveness is governed by: 

1 classically conditioned responses to stimuli that previ¬ 
ously brought pleasure or created pain mediated 
by subcortical systems and 

2 cognitive appraisals of stimuli in context mediated 
by neocortex. 

Neuroscientists have given most attention to the clas¬ 
sical conditioning studies conducted with animal 
models. Almost no attention has been given to the pos¬ 
sibility of multiple emotional systems in the brain. To 
date, the origins of fear in the amygdala have received 
attention as the dopamine-based 'reward system'. But 
interest in the explanation of neural bases of the wide 
variety of distinctly felt emotional experiences is lack¬ 
ing. With the advent of increasingly fine-grained brain 
imaging methods, this may change. 

While classical conditioning remains an important 
explanation of learned emotional associations, affec¬ 
tive neuroscientists have begun to think of mamma¬ 
lian emotion as arising from several separate genetically 
determined networks of brain areas, each serving a par¬ 
ticular adaptive function, each giving rise to a unique 
motor routine when activated, each having a unique 
'calling condition' or evoking stimulus, and each being 
the neural substrate of distinct conscious emotional 


feelings. A psycho-ethological perspective allows us to 
see emotional functioning in an adaptive context. 

2*0 PANKSEPP’S EMOTIONAL BRAIN 
SYSTEMS 

Jaak Panksepp (1998) offered a functional definition 
of an emotional system in the brain (illustrated in 
Figure 13.2): 

1 The underlying circuits are genetically predeter¬ 
mined to respond unconditionally to stimuli repre¬ 
senting evolutionary pressures faced by the species 

2 The circuits organize motor programs and auto¬ 
nomic and hormonal changes to respond to the 
environmental challenge or opportunity at hand 

3 The circuits tune sensitivities of sensory systems 
to be responsive to stimuli relevant to the emotion 
evoked 

4 The positive feedback of neural activity means 
emotional arousal outlasts the precipitating 
circumstances 

5 Emotional circuits can come under cognitive control 

6 Emotional circuits reciprocally influence higher 
decision-making and appraisal systems and 
consciousness 

7 The circuit is capable of elaborating distinctly differ¬ 
ence subjective feelings (not shown in Figure 13.2). 

Panksepp (1998) described a small set of 'hard-wired' 
emotion systems found in mammalian brains. (We will 
follow Panksepp's convention of identifying the sys¬ 
tems by capitalized labels; the capital letters remind 


Emotions control 



FIGURE 13.2 The functions of emotional systems: (1) uncondi¬ 
tioned sensory inputs, (2) coordinated physiological and behavioral 
outputs, (3) gating of inputs, (4) positive feedback, (5) cognitions 
instigating emotions, and (6) emotional control over cognitions. 
Source : Panksepp, 2006. 
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FIGURE 13.3 Four funda¬ 
mental mammalian emotional 
systems, shown with prototypi¬ 
cal behaviors. Source : Panksepp, 
1998. 


us that we are speaking of systems of emotion and not 
simply the conscious feelings associated with the sys¬ 
tems or single brain locations.) The first four emotion 
systems appear shortly after birth in all mammals: 

• SEEKING: the appetitive system that makes mam¬ 
mals curious about their world and promotes goal- 
directed behavior toward a variety of goal objects, 
such as food, shelter, sex 

• FEAR: the system which responds to pain and 
threat of destruction and leads to the well-known 
flight, fight, or freeze behavior 

• RAGE: this system mediates anger and is aroused 
by frustration, bodily irritation, or restraint of free 
movement 

• PANIC: the system that responds to separation of 
young animals from their caregiver by activating 
crying and separation calls. 

These four emotional systems and their mutually 
inhibitory or excitatory relationships with each other 
are illustrated in Figure 13.3. 


In addition to the four fundamental emotion sys¬ 
tems, three other special-purpose systems come on¬ 
line at different stages of mammalian development. 
They are: 

• LUST: the system that coordinates sexual behavior 
and feelings 

• CARE: the care giving system that is the adult 
counterpart of the infant PANIC system; CARE 
operates in both mothers and fathers and promotes 
social bonding and care giving behaviors 

• PLAY: the neural system that organizes rough-and- 
tumble play which occurs spontaneously in mam¬ 
malian young; this system supports laughter and 
may be the neural substrate of joy. 

Since each of these emotional systems has its own 
'wiring diagram', understanding emotion in the brain 
can become quite complex! 

Much of what we know about the mammalian 
emotional systems (shown in Table 13.1) comes from 
studies of non-human mammals. Studies of emotion 
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TABLE 13.1 Basic emotional systems in the brain and 
associated brain areas (adapted from Panksepp, 2006) 


Basic emotional 
system 

Associated mammalian 
brain areas 

Associated emotional 
feelings 

FEAR/anxiety 

Central and lateral 
amygdala to medial 
hypothalamus and 
dorsal PAG 

Fear, anxiety 

SEEKING/ 

expectancy 

Mesolimbic outputs of the 
VTA to the nucleus 
accumbens; mesocortical 

VTA outputs to orbitofrontal 
cortex; lateral hypothalamus 
to PAG 

Interest, curiosity 

RAGE/anger 

Medial amygdala to bed 
nucleus of the stria 
terminalis (BNST); medial 
and perifornical 
hypothalamic to PAG 

Anger, contempt 

PANIC/ 

Anterior cingulate, BNST, 

Sadness, shyness. 

separation 

distress 

and preoptic area; 
dorsomedial thalamus, PAG 

guilt/shame 

LUST/sexuality 

Corticomedial amygdala, 
BNST; preoptic area, 

VTA, PAG 

Erotic feeling, 
jealousy 

CARE/ 

nurturance 

Anterior cingulate, BNST; 
preoptic area, VTA, PAG 

Love 

PLAY/joy 

Dorsomedial hypothalamus; 
parafascicular area, PAG 

Joy, happiness 


PAG = periaqueductal gray matter is located in the interior of the 
midbrain, surrounding the cerebral aqueduct, running from the 
posterior commissure rostrally to the locus coeruleus caudally. BNTS = 
bed nucleus of the stria terminalis is a cluster of subcortical nuclei 
medial to the basal ganglia and above the hypothalamus. VTA = 
ventral tegmental area which is located in the midbrain. 

in human participants are limited by ethical con¬ 
straints on use of invasive techniques and on the 
kinds and intensities of emotional stimuli that can 
be used to evoke emotional responses. Laboratory 
studies of human emotion rely on behavioral obser¬ 
vation and neural imaging following relatively mild 
emotion-evoking tasks such as direct sensory stimu¬ 
lation and conditioning, observation of images of 
emotional events involving others, and emotion 
generation through recall and mental imagery of 
emotion-laden memories. Additional evidence comes 
from examination of neurological patients who have 
suffered lesions in various brain areas. 

2 A Feelings of emotion 

Feelings of emotion cannot be studied in non-human 
mammals, though we can make some guesses about 
the felt experiences of other mammals by observing 


their behavioral reactions. There is currently consid¬ 
erable debate about whether any evidence justifies 
speaking of 'feelings' in non-human mammals. It 
appears that there is growing sentiment in favor of 
this position based on commonality of neuroanatomy 
of emotion and of the systems underlying conscious 
experience. 

Damasio et al. (2000) studied the neural substrates 
of emotional feelings in humans. They looked for brain 
areas active during different emotional states when 
participants were asked to recall and re-experience 
emotion-laden personal memories. Using positron 
emission tomography (PET) imaging, the researchers 
located significantly different neural maps for fear, 
happiness, sadness, and anger (Figure 13.4). They 
noted different patterns of activation and deactivation 
in cortical and subcortical areas related to the repre¬ 
sentation and regulation of emotion as well as bod¬ 
ily homeostasis, including insular cortex, secondary 
somatosensory areas (SII), cingulate cortex, and nuclei 
in the brainstem and hypothalamus. 

Since these emotional experiences were inter¬ 
nally generated and did not require either percep¬ 
tion of external stimuli or facial or motor expression, 
Damasio and his colleagues argued that the resulting 
neural maps represent the neural correlates of the dif¬ 
ferent feelings of emotion and conclude: 

The neural patterns depicted in all of these structures 
constitute multidimensional maps of the organism's 
internal state, and we believe that they form the basis 
for an important aspect of mental states known as 
feelings. 


3*0 THE FEAR SYSTEM 

We will take a closer look at the FEAR system and 
the SEEKING system, as described by Panksepp: two 
emotional systems that have been studied in great 
detail and that have well-known influences on cog¬ 
nition. Next, we will broaden our focus and look at 
the fear system in humans and at the reward system, 
including emotional aspects of rewards: liking, want¬ 
ing, learning. 

The fear system is a neural system for avoiding 
pain or injury. It is based primarily in the central and 
lateral nuclei of the amygdala with connections to the 
medial hypothalamus and dorsal periaqueductal gray 
matter (PAG) of the midbrain. This system responds 
to both unconditioned stimuli (loud sounds, loom¬ 
ing and sudden movements, painful stimuli, fearful 






426 


13. EMOTION 



FIGURE 13.4 PET imagery results of participants' self-generated happiness and sadness (top row) and fear and anger (bottom row). Red 
and yellow indicate areas of increased metabolic activity; purple and blue indicate areas of decreased metabolic activity. Ob = orbitofrontal 
cortex; in = insula; bf = basal forebrain; ac = anterior cingulate; p = pons; hyp = hypothalamus; pc = posterior cingulate; SII = secondary 
somatosensory cortex; mb = midbrain. Source : Damasio et al, 2000. 


faces) and conditioned stimuli (classically conditioned 
danger signals, memories, images) arriving from the 
thalamus and sensory and association cortices (Figure 
13.5). Reciprocal efferent pathways return feedback 
signals to these thalamic and cortical sites to tune sen¬ 
sory processing in emotion-specific ways (Figure 13.6). 
It is clear that the efferent (outgoing) pathways from the 
amygdala to cortex are as complex and rich as the affer¬ 
ent (incoming) pathways from cortex to the amygdala. 

Afferent signals to the amygdala arrive via four 
pathways. Olfactory information, important for mam¬ 
mals, arrives directly at the amygdala from the olfac¬ 
tory cortex without preprocessing in the thalamus; 
this may account for the profound ability that odors 
have to evoke emotional memories. Visceral information 


reaches the amygdala from the hypothalamus and 
septal area through the stria terminalis. Affect-relevant 
information about internal states also arrives from the 
hypothalamus, thalamus, and brainstem as well as 
the orbital cortex and anterior cingulate cortex via the 
ventral pathway. Finally, sensory information arrives 
directly from temporal lobe structures such as the pri¬ 
mary auditory cortex and the hippocampus. 

The amygdala itself is a collection of nuclei and 
internal pathways that serve different functions in emo¬ 
tional processes: the basolateral complex , the centromedial 
nucleus , and the cortical nucleus. The basolateral com¬ 
plex can be further subdivided into lateral, basal, and 
accessory-basal nuclei. The lateral amygdala, which is 
afferent to the rest of the basolateral complex as well 
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FIGURE 13.5 Afferent pathways to the amygdala. Hip = hippocampus; BS = brainstem; 
Th = thalamus; Hyp = hypothalamus; A = amygdala. 



FIGURE 13.6 Efferent pathways from the amygdala. Hip = hippocampus; BS = brainstem; 
Th = thalamus; Hyp = hypothalamus; Am = amygdala. 


as the centromedial nucleus, receives input from sen¬ 
sory systems. The centromedial nucleus is the main 
output for the basolateral complex, and is involved in 
emotional arousal in mammals. The cortical nucleus 
is involved in smell and pheromone processing; it 
receives input from the olfactory bulb (Figure 13.7). 

Efferent pathways from the amygdala mirror afferent 
pathways, returning signals to subcortical locations and 


to the brainstem. Of significance for our study of cogni¬ 
tion-emotion interactions is the direct efferent pathway 
from the amygdala to entorhinal cortex, inferior tem¬ 
poral lobe cortex, and finally to visual cortex including 
the fusiform face area (Figure 13.7). There are top-down 
and bottom-up relationships between amygdala and cor¬ 
tex as they work together to tune the brain for adaptive 
responses to significant environmental threats. 
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FIGURE 13.7 The nuclei of the 
amygdala and their efferent connections 
with sensory cortex. CE = centromedial, 
L = lateral, B = basal, and AB = accessory- 
basal nuclei of the amygdala. ITC = inferior 
temporal cortex; FFA = fusiform face area; 
VI = primary visual cortex. Remember that 
there is an amygdala in each hemisphere of 
the brain. Source : Vuilleumier, 2005. 


While most of the research on the neuropsychologi¬ 
cal function of the amygdala and its functioning has 
been done on rat and other mammalian models, accu¬ 
mulating evidence from human studies has yielded 
mostly consistent results. Subcortical emotional systems 
of the brain are thought to be conserved across mamma¬ 
lian groups; the same brain structures operate in the 
same ways in diverse animals. Most researchers believe 
that findings from rats and other mammals will apply 
to human neuropsychology. However, research is ongo¬ 
ing to confirm this hypothesis. Among early human 
studies confirming the role of the amygdala in fear 
learning, LaBar et al. (1998) found that fear conditioning 
in humans resulted in an increased blood-oxygen-level- 
dependent (BOLD) signal in the amygdala as assessed 
with functional magnetic resonance imaging (fMRI) 
and that the magnitude of this BOLD response was pre¬ 
dictive of the strength of the conditioned response. 

3.1 Conscious and unconscious fear processing: 
LeDoux’s high road and low road 

LeDoux (1996) labeled the two sensory input path¬ 
ways to the amygdala for perception of fearful stim¬ 
uli the Tow road' and the 'high road'. The low road is 
a fast pathway from sensory receptor to sensory tha¬ 
lamus to the amygdala that bypasses the cerebral cor¬ 
tex (Figure 13.8). As we discussed at the beginning of 
the chapter, the direct thalamo-amygdala (low road) 
processing is only capable of low spatial resolution 


of stimuli and thus can respond only to simple stim¬ 
uli or to the gross characteristics of complex stimuli. 
This 'quick and dirty' processing enables automatic, 
unconscious reactions to the broad outlines of poten¬ 
tially dangerous stimuli before we have to time to 
think about our responses. 

The longer thalamo-cortico-amygdala pathway 
(high road) takes somewhat longer to traverse but 
allows complex, contextualized processing of stimuli 
followed by conscious, deliberate responding. The 
high road represents the pathway that is more influ¬ 
enced by social and personal decision-making pro¬ 
cesses and thus can reflect culture-specific emotional 
responses. 

3*2 Fear without awareness 

Support for the hypothesis that emotional stimuli 
can be processed via alternate conscious and non- 
conscious pathways comes from research from a 
number of studies. We will look at one conducted by 
Vuilleumeir and his colleagues (2002). They looked 
for differential neural responses to fearful and neutral 
stimuli when processed with and without conscious 
awareness in a patient with right parietal neglect and 
visual extinction due to damage in his right inferior 
parietal cortex (Figure 13.9). 

Parietal neglect is a deficit in body perception and 
visuospatial processing in individuals who have 
lesions in their parietal cortex. Patients with right 
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FIGURE 13.8 Two pathways to fear: the 
low road and the high road. 


parietal damage may have difficulty perceiving stimuli 
shown to them in the part of their visual field contralat¬ 
eral (on the opposite side) to the lesioned area - in the 
left visual field. In the laboratory, when shown single 
stimuli in the right or left visual fields, neglect patients 
have no difficulties with perception. However, when 
shown two different, simultaneously presented stimuli 
in the right and left visual fields, patients with right 
parietal neglect will frequently report not being able to 
see the stimulus presented to their left visual field. This 
phenomenon is called extinction. It provides research¬ 
ers with an opportunity to study stimulus processing 
with and without conscious perception of the stimulus. 

Vuilleumeir et al. (2002) showed their participant pic¬ 
tures of fearful faces, neutral faces, or a house, individ¬ 
ually or together, as shown in Figures 13.10 and 13.11. 
The important results involved spared processing of 
faces in the right hemisphere despite parietal damage 
and the participant's self-reported inability to see the 
stimuli. 

Vuilleumier and his colleagues found that: 

1 Fearful faces (but not neutral faces) activated the 
left amygdala, extrastriate (visual) cortex, bilateral 
orbitofrontal cortex, and right and left fusiform 
gyri when seen and when extinguished. 

2 There was no difference in processing of the fear¬ 
ful faces in the amygdala whether seen or extin¬ 
guished. Fearful faces can be processed without 
awareness despite the damaged parietal areas. 







FIGURE 13.9 Damaged right inferior parietal cortex in 
Vuilleumeir et al. (2002) participant. Source : Vuilleumeir et at., 2002. 

3 Conscious perception increased activity in the 
left fusiform, parietal, and prefrontal cortices of the 
left hemisphere compared with processing dur¬ 
ing extinction trials. Consciousness alters cortical 
processing. 

Separate results from non-lesioned participants con¬ 
firm that attention to stimuli (for both task-relevant and 
non-task-relevant stimuli) and the emotional signifi¬ 
cance of stimuli (fearful or neutral) make independent 
contributions to visual perception (Vuilleumier et al., 
2001). Activity in the fusiform gyrus of the ventral 
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FIGURE 13.10 Unilateral stimuli 
used by Vuilleumeir et ah (2002). When the 
participant fixated on the light square, the 
figures projected either to the contralateral 
right or left hemisphere. He was able to 
see these figures 100 percent of the time. 
Source : Vuilleumier et ah, 2002. 


Bilateral 



Face perceived 




Face extinguished 



Face perceived 




Face extinguished 




FIGURE 13.11 Bilateral stimuli. 
When the participant fixated on the 
light square, he reported being unable 
to see the faces (projected to his dam¬ 
aged right cortex) about 65 percent of 
the time. Source : Vuilleumier et ah, 2002. 


temporal lobe is greater for attended stimuli than for 
non-attended stimuli; fusiform activation is greater for 
fear faces regardless of attentional level (Figure 13.12). 

The findings support the idea that there are inde¬ 
pendent conscious and unconscious pathways to the fear 
processing system of the amygdala. 


3.3 Affective blindsight 

We have known for some time (Weiskrantz, 1986) that 
patients with lesions in their visual cortex have a pre¬ 
served ability to respond to the visual features of objects 
presented in the corresponding blind visual field. Even 
though such patients have no awareness of the stimuli 
and cannot describe them, they can, nevertheless, react 
in behaviorally appropriate ways to specific objects, i.e. 
they can make appropriate grasping movements with 
their hands, they can 'guess' at above chance levels 
the direction of movement, orientation, and color of 
objects. This ability has been called blindsight. Recently, 
researchers have begun to simulate blindsight in the 
laboratory by using transcranial magnetic stimulation 


(TMS) temporarily to interrupt processing in the visual 
cortex (Ro et al., 2004). This research seems to confirm 
the long-held belief that such non-conscious visual 
processing depends on sensory abilities of the evolu- 
tionarily ancient superior colliculus. 

The term affective blindsight has been coined to refer 
to the preserved ability of patients with visual corti¬ 
cal lesions to respond to the affective qualities of stimuli 
shown to them in their blind visual field (de Gelder 
et al., 2000; Heywood and Kentridge, 2000). 

de Gelder et al. (2005) were able to study the inter¬ 
action of conscious and unconscious emotional 
processing in a participant with extensive left visual 
cortex lesions by presenting facial expressions simul¬ 
taneously to the intact and blind visual fields. By 
examining the participant's report of consciously 
experienced stimuli, the researchers could look for 
effects of the non-conscious stimuli on conscious 
reports. When the conscious and non-conscious fear 
stimuli were congruent (fear seen face with a fear 
unseen face), correct identification of facial expression 
was high. However, for incongruent faces (happy seen 
face paired with fear unseen face or vice versa), the 
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Attended Unattended 

faces faces 


FIGURE 13.12 Activity in the fusiform face area of the tempo¬ 
ral lobe: contributions of attention and emotion. Top left : Coronal sec¬ 
tion showing fusiform activity. Top right : Horizontal section showing 
fusiform activity. Bottom: Effects of attention and emotion on level of 
fusiform activation. Source: Adapted from Vuilleumier et al., 2001. 

correct identification dropped off to chance level. In 
congruent fear face trials, there was increased activity 
in the left amygdala, leading researchers to conclude 
that unconscious processing of unseen fear faces influ¬ 
enced conscious processing and that this effect was 
mediated by the amygdala and superior colliculus. In 
an additional experiment, the researchers looked at 
the influence of seen and unseen faces on perception 
of an emotional voice (happy or fearful). For congru¬ 
ent fear face and fear voice, perception of the voice 
was much more accurate than when the face and 
voice were incongruent - irrespective of whether the 
face was consciously or non-consciously processed. 
Unseen happy faces and emotional pictures other than 
faces did not have similar enhancing effects. Overall, 
the researchers concluded that recognition of fear is 
mandatory and independent of awareness. There are 
separable cortical and subcortical pathways for per¬ 
ception of fearful stimuli. 

3*4 Cognition-emotion interactions 
in fear responses 

Now that we understand the basic wiring diagram of 
the amygdala's inputs and outputs, we can begin to 


examine cognition-emotion interactions that depend 
on the amygdala and its associated areas. 

A recent review of the role of the amygdala in emo¬ 
tional processing (Phelps and LeDoux, 2005) identified 
five areas in which there is converging evidence from 
animal and human studies of cognition-emotion inter¬ 
actions involving the amygdala: 

1 implicit emotional learning and memory 

2 emotional modulation of memory 

3 emotional influences on perception and attention 

4 emotion and social behavior 

5 emotion inhibition and regulation. 

We will look at paradigmatic studies involving human 
participants. 

3*5 Implicit emotional learning and memory 

Implicit memory has been discussed in an earlier 
chapter. It is a kind of learning that is demonstrable in 
behavioral indicators but that cannot be recollected or 
consciously reported. It includes various kinds of pro¬ 
cedural knowledge, grammars of languages, and clas¬ 
sically conditioned associations. Implicit emotional 
memory involves retention of classically conditioned 
emotional relationships that cannot be voluntarily rec¬ 
ollected or reported. One kind of evidence comes from 
patients with various kinds of neurological damage. 

Patients with damaged amygdalas but spared hip¬ 
pocampi fail to show the physiological indicators of 
fear conditioning (heart rate increases, electrodermal 
responses, etc.), but they can recollect and report epi¬ 
sodic memories of the circumstances around the fear 
conditioning. Conversely, patients with lesions in the 
hippocampus where the amygdalas are spared are 
unable consciously to report the events surrounding 
the fear conditionally but show normal fear condition¬ 
ing measured physiologically (Bechara et ah, 1995). 
This dissociation between the physiological expres¬ 
sion of fear conditioning (amygdala-dependent) and 
the episodic reporting of events surrounding condi¬ 
tioning (hippocampus-dependent) suggest that there 
are multiple systems supporting emotional memories. 

Clinical psychologists who work with victims of 
traumatic events (accidents, abuse, combat) also have 
an interest in implicit emotional memory. 

3*6 Emotional modulation of explicit memory 

Psychological evidence has been available for some 
time indicating that moderate levels of emotional 
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arousal (most often fear-based arousal) at the time of 
an event lead to better retention of explicit memories. 
For example, frightening films are better remembered 
than neutral films, with a linear relationship between 
degree of emotional arousal in the film (measured by 
self-report and by PET activity in the amygdala) and 
level of free recall (Cahill etal., 1996). The familiar 
inverted U-shaped function is in operation in predict¬ 
ing the interaction of emotion and memory consoli¬ 
dation. Too much activation in the amygdala leads to 
loss of explicit memories for emotional events (Cahill 
and McGaugh, 1998). 

Pathways to consolidation of explicit memories 
seem to depend on reception of emotional stimuli by 
the amygdala, followed by activation of the hypotha¬ 
lamus and pituitary gland, resulting in release of the 
adrenomedullary hormone adrenaline and subse¬ 
quently the adrenocortical hormone cortisol. Both 
adrenaline and cortisol (often called stress hormones) 
appear to influence the hippocampus-dependent for¬ 
mation of explicit memories (Figure 13.13). Improved 
memory for emotional stimuli is absent in patients 
with bilateral amygdala lesions (Cahill etal., 1995). 
The arousal effect due to stress hormone release can 
also be eliminated by administering (3-adrenergic 
antagonists ('beta blockers' such as propanolol) that 
block adrenaline receptors (Cahill etal., 1994). The 
hyper-responsivity of post-traumatic stress disorder 
(PTSD) patients can be ameliorated by administration 
of propanolol in the emergency room soon after trau¬ 
matic events (Pitman et ah, 2002). 


3.7 Emotional influences on perception 
and attention 

One of the defining features of emotional systems, 
according to Panksepp's criteria (presented at the 
beginning of the chapter), is that emotional circuits can 
influence higher level processing. Evidence for such 
influence has been obtained in non-human mammals 
where fear conditioning was found to alter the neural 
representation of the conditioned stimulus (CS) in the 
auditory cortex. Studying the guinea pig, Bakin et ah 
(1996) found that classical conditioning of tones shifts 
the tuning frequency of individual cortical neurons 
toward the frequency of the CS. The researchers report 
that such receptive field plasticity is associative, highly 
specific, rapidly acquired, and indefinitely retained. 

Parallel but more limited findings have been 
obtained for human participants. As we have seen 
above, the amygdala has direct and indirect connections 



FIGURE 13.13 Stress hormones and explicit memory consoli¬ 
dation. Adrenaline pathway: green. Cortisol pathway: red. Notice 
that both pathways begin in the amygdala, circulate to the adrenal 
gland, and feed back to the amygdala and hippocampus after pass¬ 
ing throughout the body. 

to sensory cortices that are possible pathways for 
positive feedback to perceptual processes. Imaging 
studies using visual masking methods and subliminal 
fear conditioning (Morris et ah, 2001) have shown that 
human participants who are subliminally exposed to 
face stimuli as CS for an aversive loud noise (UCS) 
show an increased responsiveness in the amygdala 
and visual cortex over trials to the CS. The researchers 
concluded that 'the parallel learning-related responses 
observed in ventral amygdala and visual cortex are 
consistent with a proposal, therefore, that "feed¬ 
back" efferents from basal amygdala nuclei mediate 
emotion-dependent modulation of visual processing'. 

The amygdala appears to play a role in determining 
how unattended but significant stimuli gain access to 
consciousness by temporary feedback to cortical areas 
involved in receiving sensory input. The action of the 
amygdala can make the cortical areas momentarily 
more receptive to certain adaptively important stimuli. 

An example of this greater receptivity to significant 
stimuli can be found in the Stroop test (Box 13.1) when 
personally emotion-evoking words are compared with 
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BOX 13.1 Emotional Stroop test 


A version of the emotional Stroop test that influenced responding by Vietnam veterans (adapted from McNally et al., 
1990) 


Controls 

War-related 

Negative 

Neutral 

Positive 

OOOOOOO 

firefight 

germs 

mix 

loyal 

OOOOOOO 

Medevac 

filth 

millionaire 

pleasant 

OOOOOOO 

Nam 

dirty 

concrete 

happy 

OOOOOOO 

bodybag 

urine 

input 

friendship 


other positive, negative, and neutral words. The result 
has been obtained with Vietnam veterans (McNally 
et al., 1990) as well as civilian trauma survivors (Taylor 
etal., 2006). Words that have personal significance 
for participants (body bags, 'Nam, Medevac for the 
Vietnam veterans versus revolver, incest, 9/11, or fire for 
civilian victims) are repeatedly found to gain access to 
awareness more readily than neutral words and, conse¬ 
quently, cause greater delay in naming the color of the 
ink in which the words are printed. However, patients 
who have suffered bilateral damage to the amygdala 
fail to show the expected facilitation of attention for 
emotional words than is found in normal participants. 
Though participants with amygdala lesions compre¬ 
hend the meaning of the words, they do not display 
selectively enhanced perception of verbal stimuli with 
aversive content (Anderson and Phelps, 2001). 


3*8 Emotion and social behavior 

Bilateral damage to the amygdala in monkeys causes 
dramatic changes in their behavior, including lack of fear 
for dangerous stimuli, eating inedible objects, and atypi¬ 
cal sexual behavior (Kluver-Bucy syndrome). However, 
parallel behavioral changes have not been noted in 
human patients with lesions of the amygdala, presum¬ 
ably because humans have extensive neocortical social 
control and inhibition systems not available to other pri¬ 
mates. There is, however, some evidence that more sub¬ 
tle deficits exist in patients with amygdala damage, for 
example, in processing of emotional facial expressions. 

As we noted at the beginning of the chapter, the 
amygdala responds to simple aspects of complex 



FIGURE 13.14 Amygdala responds to the whites of fear¬ 
ful eyes more strongly than to happy eyes. On the right, the BOLD 
response of the left ventral amygdala to fearful whites. Source : 
Whalen et al, 2004, as adapted in Phelps and LeDoux, 2005. 


stimuli, specifically to low spatial frequency aspects 
of faces (Vuilleumeir et al., 2003). Consistent with 
this general rule, evidence shows that the amygdala 
responds to the wide-open eyes of fearful and sur¬ 
prised expressions. In a recent refinement, Whalen 
and his colleagues (Whalen et al., 2004) found that the 
amygdala responded selectively to subliminally pre¬ 
sented whites of fearful eyes compared to the whites of 
happy eyes (Figure 13.14). 

Finally, Adolphs et al. (2005) found that patients 
with bilateral amygdala damage fail to look at the 
eyes when judging facial expression. This deficit may 
account for observed difficulties in interpretation of 
others' emotion among such patients. Patients with 
amygdala lesions underestimate emotional intensity 
and overestimate trustworthiness and approachable¬ 
ness of others compared to non-lesioned participants. 
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3*9 Emotion inhibition and regulation 

Fear learning is a long-lasting and stable kind of learning 
that is remarkably resistant to change and voluntary con¬ 
trol. It is very difficult to think one's self out of a fearful 
response. While fear learning is adaptive in that it allows 
us to avoid predictable dangers in the environment, it 
can become disabling if it is misplaced or exaggerated. 
Research on modification or elimination of emotional 
associations by extinction or reversal, reconsolidation, 
and emotion regulation is of considerable importance. 

Extinction is a method of Pavlovian conditioning 
where a conditioned stimulus (CS) that was previously 
linked to an aversive unconditioned stimulus (UCS) is 
presented alone for a number of trials. The participant 
learns that the CS is no longer a signal for the aversive 
event in that context. The CS will continue to evoke the 
fear response after the passage of time, in other loca¬ 
tions, with re-exposure to the UCS. This shows that 
the learned fear response has been retained in memory 
and that extinction learning operates by inhibiting the 
fear response. 

The neuroscience of extinction is well studied. It 
depends critically on the activity of NMDA receptors 
in the amygdala. When NMDA receptors are blocked 
in rats, extinction learning is disrupted. When activity 
at NMDA receptors is enhanced, extinction is aug¬ 
mented. Recently, researchers (Ressler et al. r 2004) used 
NMDA agonists for the first time to improve extinc¬ 
tion learning in human agoraphobic patients. They 
found that two doses of D-cycloserine significantly 
reduced agoraphobia in conjunction with exposure 
therapy compared to therapy with a placebo, assessed 
by self-report and by electrodermal responses to the 
UCS. The improvements were maintained at three 
months post-treatment. These results suggest possi¬ 
bilities for future treatments of negative emotion that 
rely on the neuroscience of fear in the amygdala. 

Reversal conditioning is similar to extinction learning in 
that it modifies the contingencies between conditioned 
stimuli and responses. In this paradigm, participants 
are first conditioned to fear one stimulus (CS+) which 
is reliably followed by a fear-inducing stimulus (UCS), 
such as a loud noise, and not to fear another stimulus 
(CS-) which is never followed by the UCS. In the next 
phase, the contingencies are reversed so that the previ¬ 
ously neutral stimulus is now paired with the UCS. 

Morris and Dolan (2004) used reversal condition¬ 
ing to explore the role of the amygdala and the orbit- 
ofrontal cortex in fear conditioning. They used a 
neutral facial expression as CS+ with a loud noise 
as UCS. They found that during initial conditioning. 


strong bilateral activation in the amygdala developed 
to the CS+. After reversal, the new face CS elicited 
enhanced responses in the orbitofrontal cortex while 
the old CS continued to evoke increased responses in 
the right ventral amygdala. While the orbitofrontal 
cortex is capable of rapid reversal of fear responses, the 
amygdala showed a persistent, non-reversing response 
to previous fear-related stimuli. Cortex and limbic 
system seem to follow different programs - cortex is 
rapidly responsive to new contingencies while limbic 
system is more conservative and retains old triggers. 

This finding speaks to the neurological basis of the 
experience of 'being of a divided mind' where part of 
us knows that the danger no longer exists and part 
of us still responds to old cues. Those who live with 
post-traumatic stress disorder (PTSD) are quite famil¬ 
iar with this non-voluntary kind of responding despite 
changed circumstances. 

Memories, including fear memories, become perma¬ 
nent through a process of protein synthesis called con¬ 
solidation. When retrieved, the memory again becomes 
labile and is susceptible to further manipulation and 
alteration prior to reconsolidation. Evidence shows that 
reconsolidation of fear memories in rats involves addi¬ 
tional protein synthesis in the amygdala (Nader et al., 
2000). Infusion of an antibiotic that interrupts protein 
synthesis eliminates the conditioned response at test the 
next day. To date, no direct studies of protein synthesis 
and reconsolidation involving human participants have 
been published. It is a promising line of future research 
with the potential to alleviate PSTD symptoms. 

Finally, emotional responsiveness can be regulated 
to some extent by top-down influences from cortex to 
amygdala. Reappraisal and reinterpretation of events are 
primary coping strategies for human beings. We can 
reappraise circumstances and attempt to see them in 
a different way with different meanings and implica¬ 
tions for us. For example, we can reinterpret taking a 
wrong turn from an inconvenience and time-waster to 
an opportunity to explore new territory. We thereby 
change our emotion from frustration to curiosity. 
Reappraisal can lead to both down-regulation as well 
as up-regulation of affect. Such reappraisal alters 
our subjective experience as well as our physiologi¬ 
cal responses. In general, reappraisal strategies acti¬ 
vate prefrontal and anterior cingulate cortex involved 
in cognitive control and the amygdala increasing or 
increasing activation in parallel with emotional arousal. 

Ochsner and his colleagues (Ochsner etah, 2004) 
looked for fMRI correlates of reappraisal of pictured 
negative events. They looked for areas of brain activa¬ 
tion when participants up-regulated or down-regulated 
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FIGURE 13.15 Up and down 
regulation. Activations for appraisal 
strategies compared to looking at pic¬ 
tures without appraisal, (a) Activation 
related to up-regulation of negative 
affect. Lateral views are shown on the 
left and center; medial views on right. 
Note left dorsal lateral and medial 
prefrontal as well as anterior cingulate 
cortical activation, (b) Views for down- 
regulation. Note bilateral activation 
of lateral and medial prefrontal cor¬ 
tex, including many left-sided regions 
similar to those used when increasing 
affect. Source : Ochsner et al., 2004. 


negative affect related to pictured negative events, 
such as a sick person in the hospital. Participants were 
assigned one of two kinds of appraisal strategies. In 
the self-focus group , they were asked to focus inter¬ 
nally on the meaning of the negative event - either up- 
regulating the negative affect by imagining themselves 
or someone they love as the person in the picture or 
down-regulating by increasing personal distance from 
the event by seeing it from a detached objective per¬ 
spective. In the situational focus group , participants were 
asked to consider aspects of the situation either imag¬ 
ining that things would get better (down-regulating 
negative affect) or imagining that things would get 
worse (up-regulating negative affect). 

Both up- and down-regulating negative emotion 
recruited prefrontal and anterior cingulate regions com¬ 
monly implicated in cognitive control. Amygdala activa¬ 
tion was modulated up or down in accord with appraisal 
strategies used. Up-regulation uniquely recruited regions 
of left rostromedial PFC implicated in the retrieval of 
emotion knowledge, whereas down-regulation uniquely 
recruited regions of right lateral and orbital PFC impli¬ 
cated in behavioral inhibition (Figure 13.15). 

Results also indicated that self-focused regulation 
recruited medial prefrontal regions (BA 32) implicated 
in internally processing and self-referential judg¬ 
ments. Situation-focused regulation recruited lateral 
prefrontal regions (BA 46) implicated in maintenance 
and manipulation of information about the external 
world via working memory (Figure 13.16). 

These data suggest that both common and distinct 
neural systems support various forms of reappraisal. 
Particular prefrontal systems modulate the amygdala 


in different ways depending on the regulatory goal 
and strategy employed. 

We can see that the fear system that gives rise to 
negative emotion has numerous interactions with 
cognitive systems in the cortex. We turn now to the 
system that Panksepp called the SEEKING system, 
another highly studied emotional system in the brain. 
In human studies, these aspects of emotional process¬ 
ing are typically referred to as reward systems or path¬ 
ways, and correspond to human emotions of liking 
and wanting. We will look at a few examples of cogni¬ 
tion-emotion interactions involving the reward system. 


4*0 THE REWARD SYSTEM: LIKING, 
WANTING, LEARNING 

To review, Panksepp (1998) described the SEEKING 
system as the appetitive system that makes mammals 
curious about their world and promotes goal-directed 
behavior toward a variety of goal objects, such as 
food, shelter, sex. This concept of the SEEKING sys¬ 
tem includes classical reward pathways in the brainstem 
as well as other subcortical areas. In contrast with the 
FEAR system which gives rise to freezing, hiding, or 
flight in service of self-protection when activated, the 
SEEKING system is a positively valenced, energizing 
system that moves animals out into their environment 
to forage and explore. 

In 1954, Olds and Milner discovered that rats 
would learn to work for electrical stimulation of the 
subcortical areas including areas around the lateral 
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FIGURE 13.16 Situation-focused regulation. Activations unique to appraisal strategies when down-regulating affect. Situation focus on 
left: notice activity in BA 46 (circled). Self-focus on right: note activity in BA32 (circled). Source : Ochsner et at., 2004. 


hypothalamus. Subsequent studies have shown 
this electrical self-stimulation (ESS) is highly rein¬ 
forcing. Rats will self-stimulate areas in the lateral 
hypothalamus up to 2000 times an hour and choose 
ESS over food to the point of starvation. Development 
of microelectrodes in the 1970s allowed increasingly 
fine-grained studies of brain locations involved in 
reward. 

The primary reward pathways in mammals 
include the: 

1 mesolimbic dopamine pathway: dopaminergic 
neurons that originate in the ventral tegmental area 
(VTA) of the brainstem terminate at the nucleus 
accumbens in the forebrain and 

2 mesocortical dopamine pathway: dopaminergic 
neurons project from VTA to orbitofrontal cortex 
(Figure 13.17). VTA is located inside the pons in the 
midbrain, surrounded by other midbrain nuclei 
and pathways. Nucleus accumbens is located near 
and anterior to the amygdala and below the stria¬ 
tum (basal ganglia) in each hemisphere. 

4.1 Re-interpreting ‘reward’: from reward to 
reward prediction to reward prediction 
error 

Understanding of the reward concept and the role of 
midbrain dopamine pathways has evolved. For a long 
time, the dopamine system was thought to respond 
at receipt of rewarding stimuli, such as ESS, intracra¬ 
nial drug injections, food, or sweet water or to condi¬ 
tioned stimuli that predicted the reward. Subsequent 



FIGURE 13.17 Dopamine reward pathways including the 
mesocortical dopamine and mesolimbic dopamine pathways. 


research has shown that neurons in this system are 
more responsive to anticipation of reward than to receipt 
of reward (Schultz, 1998, 2002). Evidence in support of 
this hypothesis shows that dopamine neurons: 

1 fire selectively in response to unexpected rewards 
and novel, attention-grabbing events, followed 
by a rapid decrease in responding with repeated 
arrival of reward as expected 

2 stop responding to predictable rewards, as they 
gradually become responsive to conditioned stim¬ 
uli that predict rewarding events 

3 are inhibited by the omission of expected rewards. 
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The reward system begins with unconditioned 
responses to sensory aspects of rewarding objects: 
unconditioned stimuli (UCS) such as sweet tastes, 
musky smells, etc. Since the primary reinforcing event 
is actually the arrival of dopamine at the nucleus 
accumbens, the UCSs are not the rewards. They are 
proximal cues of upcoming internal events. With 
experience, more distal, environmental signals such as 
landmarks and sounds (conditioned stimuli, CS) come 
to predict the availability of taste and smell-carrying 
objects. Now these new cues come to evoke dopamin¬ 
ergic responses in the midbrain. Over time, it is 
the acquired reward stimuli that seem to produce 
the greatest pleasure. In our own lives, we are more 
excited about the prospect of a pay raise or a hot date 
than we are at 'receipt' of these desired events (Wise, 
2002). Sadly, at least from the point of view of the 
dopamine system, there seems to be something to the 
idea that the desired object, once attained, is no longer 
as desirable. 

The simple reward prediction view has been 
revised in light of evidence suggesting that dopamin¬ 
ergic neurons respond specifically to errors in predic¬ 
tion of rewards. For example, over time, activation 
disappears to CSs that reliably predict reward. The 
loss of response is not due to a generalized loss of 
response to the reward itself, since the dopaminer¬ 
gic system responds to the rewards outside the test 
situation. Also, response of dopaminergic neurons 
is depressed when rewards are withheld or delayed 
when CSs indicate that they should arrive. Since there 
are no other novel stimuli present at the time when 
rewards are expected, the depressed response cannot 
be interpreted as a response to another stimulus. The 
depressed dopaminergic response when rewards are 
withheld is seen as a reaction to the failed arrival of a 
predicted event. 

The evidence suggests that dopamine neu¬ 
rons shift responsiveness toward situations where 
rewards are highly unpredictable. In fact, CSs can¬ 
not be conditioned to rewards that are already reli¬ 
ably predicted. The long-standing empirical test 
for this hypothesis is the blocking paradigm. In the 
blocking paradigm, it can be demonstrated that a 
new stimulus will not become associated with a rein¬ 
forcer when it is presented in a circumstance where 
the reinforcer is already predicted 100 percent of the 
time. Direct empirical evidence that the dopamin¬ 
ergic system is involved in mediating this behav¬ 
ioral learning has been obtained in monkeys where 
microelectrodes record responses of individual 
dopaminergic neurons. 



-1 0 1 
Stimulus 


FIGURE 13.18 An example of an individual dopamine neu¬ 
ron that is inhibited by a stimulus predicting reward omission. At 
the top of each panel is a cumulative histogram of action potential 
number across time (each bar is 10 ms) for repeated presentations 
of the stimuli; below each histogram, the dashes represent individ¬ 
ual action potentials occurring in each trial. Stimulus 'A' is paired 
with reward (A+) and excites the dopamine neuron. When 'A' is 
presented with 'X' no reward occurs (AX—), and therefore 'X' pre¬ 
dicts reward omission. When 'X' is presented alone, the dopamine 
neuron is inhibited (X—). Other control stimuli were also presented 
but are not shown here. Source: Ungless, 2004; this figure was repro¬ 
duced from Tobler et al, 2003, with permission. 

A study by Tobler et al. (2003), reviewed by Ungless 
(2004), showed that dopamine neurons are not acti¬ 
vated by all salient stimuli. They trained monkeys 
by rewarding them every time a light was turned 
on. Then the light came on, a tone was simultane¬ 
ously sounded and no reward was delivered (see 
Figure 13.18); thus the tone predicted the omission 
of a reward. When the tone was presented alone, the 
dopamine neuron was inhibited. Ungless (2004) sug¬ 
gested that these findings indicate that dopamine neu¬ 
rons are not activated in a non-specific manner when 
salient stimuli are presented: it is the stimuli's predic¬ 
tive powers and not just the presence of a reward that 
modulate dopamine neuron activity. 

Currently, reward prediction error theory is the 
most widely accepted understanding of the dopamin¬ 
ergic reward system. 













































































































438 


13. EMOTION 



pallidum tegmentum 

FIGURE 13.19 Simplified view of subcortical liking and wanting pathways, shown in a rat brain. 'Liking' 
pathways are shown in green; 'wanting' pathways in yellow; cognitive processing of cues is shown in blue. Source : 
Berridge and Robinson, 2003. 


4.2 Reward is more than learning 

The reward prediction error theory of reward focuses 
heavily on the learned aspects of rewards. Quoting 
Thorndike's early view, Schultz (2002) emphasized 
the function of rewarding events in 'stamping in' 
associations between conditioned stimuli and their 
consequences. In response, it has been noted by 
other researchers (Berridge and Robinson, 1998, 2003; 
Panksepp, 2005) that reward prediction theory leaves 
out important hedonic and motivational components 
of rewarding events that should not be ignored. 

The hedonic feeling of 'liking' has been shown to 
have reliable facial characteristics across different 
mammals and to be dissociable from the dopamine 
system. 'Liking' reactions are neurally modulated by a 
distributed brain network that includes the shell of the 
nucleus accumbens, ventral pallidum , and brainstem para- 
brachial nucleus (Berridge and Robinson, 2003; Figure 
13.19). Liking reactions to sweet water in rats are not 
affected by activation or suppression of the mesolim- 
bic dopamine systems but are increased by injections 
of opioid- or GABA-agonist microinjections. Lesions 
that eliminate dopamine in the nucleus accumbens 
and produce a profound aphagia (disinclination to 
eat) fail to disrupt taste 'liking' (Wyvell and Berridge, 
2000). Dopamine receptor antagonists often fail to 
suppress subjective pleasure ratings of amphetamines 
and cigarettes in humans. Finally, activation of the 


human dopamine system by amphetamine correlates 
better with subjective ratings of wanting for drug or 
food than with subjective ratings of pleasure (Leyton 
et ah, 2002). In subjective terms, the dopamine system 
makes food and drugs more desired and sought out 
but does not make them palatable, tasty, or enjoyable 
once acquired. Likeability apparently depends on a 
separate system involving the shell of the nucleus 
accumbens. 

It makes some sense in light of this evidence to sep¬ 
arate the learning aspects of reward, i.e. the ability of 
rewarded behaviors to be retained, from the hedonic 
liking aspects. Homologies in behavioral responses to 
tastes give us indicators of the hedonic impact of stim¬ 
uli (Berridge, 2000). Homologous indicators of 'liking' 
across species of mammals include tongue protrusion 
for tastes such as sweet water. Conversely, disliking 
reactions include open-mouthed gapes to bitter tastes 
like quinine (Figure 13.20). 

A third component of the reward architecture is the 
element of wanting or incentive salience (Berridge and 
Robinson, 1998). Wanting is separable from liking and 
learning subjectively and neurologically (Figure 13.21). 
We can want things, such as drugs of abuse or ciga¬ 
rettes, even when we don't like them. Learning may 
be motivated by wanting but can take place without 
wanting. Likewise, learning can take place without 
subjective 'liking' as when receiving intracranial drug 
injections for learning lever-pressing responses without 
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an accompanying sensory experience of a reinforcer. 
Wanting 'transforms mere sensory information about 
rewards and their cues (sights, sounds, and smells) into 
attractive, desired, and riveting incentives' (Berridge & 

'Liking' expression - sweet 


'Disliking 1 expression — bitter 


FIGURE 13.20 Liking reactions: objective indices of hedonic 
aspects of emotion. Homologous affective facial expressions by 
infant human, juvenile orangutan, and adult rat to diked' sucrose 
(top) versus 'disliked' quinine (bottom). Source : Berridge and 
Robinson, 2003. 


Robinson, 2003 p. 510). In contrast to the liking com¬ 
ponent of reward, wanting, or in humans, craving 
is highly dependent on the mesolimbic dopamine 
pathway. 


4.3 ‘Reward pathway’ and drug use 

All drugs of abuse have their effects on the 'wanting' 
system via the dopamine pathways and their con¬ 
nections in the midbrain (Figure 13.22). Animals will 
learn to press a lever to receive intracranial microin¬ 
jections and acquire conditioned place preferences 
for locations where they receive such microinjection 
of drugs. The behavior of these animals has helped to 
isolate brain areas involved in the rewarding effects of 
cocaine, amphetamine, nicotine, alcohol, and opiates 
(Spanagel and Weiss, 1999; Ikemoto and Wise, 2004). 
In addition, drug effects in humans have been studied 
by investigating the effect of drug agonists and antag¬ 
onists on the experience of drugs of abuse by drug¬ 
using individuals. 

Natural rewards and drugs of abuse act in differ¬ 
ent ways on the mesolimbic dopamine system. Food, 
as well as opioids, indirectly affects the reward system 
by decreasing the action of inhibitory interneurons 
that normally inhibit the dopaminergic neurons of the 
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FIGURE 13.21 A schematic of psychological components of Wanting, Learning, and Liking 
and the measurable behaviors used to investigate them. Source : Berridge, 2009. 
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feedback 

FIGURE 13.22 Drugs and their effects on the 'wanting' sys¬ 
tem. Direct and indirect activation of VTA dopaminergic activity by 
natural rewards and drugs in the mesolimbic dopamine system. A10 
dopaminergic neurons originate in the VTA and project to the shell 
of the nucleus accumbens (NAC). GABA-ergic interneurons within 
the VTA and long-loop GABA-ergic feedback from the NAC pro¬ 
vide inhibitory control (minus signs) of the A10 neurons. Different 
dopamine receptors in the NAC ( D 1 and D 2 ) mediate reward effects. 
Source : Spanagel and Weiss, 1999. 

VTA; the result is increased activity in the dopamin¬ 
ergic neurons of the VTA. In contrast, cocaine and 
amphetamine act directly in the nucleus accumbens 
to maintain high levels of dopamine at synapses of 
dopaminergic neurons; cocaine inhibits reuptake 
while amphetamine apparently increases dopamine 
release. The specific modes of action of various drugs 
are still being investigated. However, it is clear that 
the agents outlined in Table 13.2 affect behavior and 
subjective experience through their effects on the mes¬ 
olimbic dopamine system (Spanagel and Weiss, 1999; 
Ikemoto and Wise, 2004). 

4 A Reward cues influence attention 

Consistent with the reward prediction error view, 
cues that predict availability of drugs of abuse quickly 
become conditioned stimuli and powerful evokers of 
craving in humans. Drug-related cues not only activate 
the dopaminergic system and create the subjective 
experience of craving but they also influence cognitive 
activities. Use of drugs shifts attention toward drug- 
related cues at the expense of other stimuli. Numerous 
studies using the Stroop paradigm (discussed ear¬ 
lier) have shown that drug-related cues (pictures 
and words) divert attention from the color-naming 
task in drug-using participants but not in non-using 


TABLE 13.2 Agents and their effects on behavior: sites 
implicated in the rewarding effects of natural rewards and 
drugs of abuse 


Agent 

Effective site 

Food, liquids. 

Cause dopamine release in VTA through inhibition 

sex 

of GABA-ergic interneurons that normally inhibit 
the dopaminergic neurons 

Mu and delta 

Opioids act indirectly through inhibition of GABA- 

opioids 

ergic neurons in VTA and NAC that normally inhibit 
the dopaminergic neurons 

Amphetamine 

Acts directly to release dopamine in the nucleus 
accumbens 

Cocaine 

Acts directly to block the dopamine transporter 
system/re uptake in the shell of the nucleus 
accumbens 

Phencyclidine 

Acts directly in the shell of the nucleus accumbens 

(PCP) 

through blockade of NMDA receptors 

Cannabis 

Acts in the posterior VTA and shell of the nucleus 
accumbens 

Ethanol 

Acts indirectly through posterior VTA through 
inhibition of the GABA-ergic interneurons 

Nicotine 

Nucleus accumbens/ventral striatum 


participants (Hester et al., 2006). Corresponding brain 
imaging studies have shown that smokers have sig¬ 
nificantly greater responses to smoking-related pic¬ 
ture cues in the nucleus accumbens and neighboring 
ventral striatum than non-smokers (David et al., 2005), 
though there is no difference between the groups' 
imaging responses to neutral images. 


5*0 SUMMARY 

Mammals have separate emotional systems in the 
brain, each with patterned, innate responses to stim¬ 
uli in the expected environment of the species related 
to survival. Systems such as the fear system and the 
reward system have been shown to have both uncon¬ 
ditioned and conditioned responses to significant 
'calling conditions' supported by separate neural net¬ 
works: fear relying on the amygdala and its connec¬ 
tions, and the reward system relying heavily on the 
mesolimbic and mesocortical pathways of the VTA. 
Each of these systems can come under cognitive con¬ 
trol and also reciprocally influence higher decision 
making, appraisal systems, and consciousness. Each 
system is capable of elaborating distinctly different 
subjective feelings. 
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6*0 CHAPTER REVIEW 6*2 Drawing exercises 

1 Label the regions of the triune brain. 

6♦ 1 Study questions 

1 What are the key differences between the reptil¬ 
ian and the mammalian brain? 

2 What are examples of 'hard-wired' emotions as 
described by Panksepp? 

3 Why has the fear system been more studied 
than other emotion systems? 

4 What do the terms 'high road' and 'low road' 
refer to in terms of fear processing? 

5 In what ways has the study of affective blind- 
sight informed us about emotion processing? 

6 What factors distinguish 'reward' from 'learn¬ 
ing' in the study of SEEKING? 


FIGURE 13.23 


2 Label the afferent connections to the amygdala. 




FIGURE 13.24 
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Man is a social animal. 


Attributed to Aristotle 



If this looks like a scene we could imagine with humans, it is because our social 
brains have large areas of overlap and similarity with the brains of other pri¬ 
mates. The two cortical hemispheres illustrated above show how social cognition 
often activates the superior temporal sulcus (STS) in the human brain. Source : Top: 
de Waal, 2004; bottom: Allison et al, 2000. 
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1.0 OVERVIEW 

In this chapter, we will examine one aspect of human 
cognition that makes us unique among our mammalian 
relatives - our ability to understand each other as con¬ 
scious beings with internal mental states. Other mam¬ 
mals have partial abilities to tune into the psychological 
states of members of their own species, their conspecif- 
ics. They can perceive aspects of posture, vocalization, 
and facial expression as signs and take action based 
on those signs. Some primates have 'mirror neurons' 
that appear to allow them to register the commonality 


between actions of other monkeys and their own 
actions. However, no other animal has been shown to 
understand, to make use of, or to depend on the subjec¬ 
tivity of other members of its species. It is a peculiarly 
human ability. 

The 'question of other minds' is an old one among 
philosophers. However, psychological and neuroscien¬ 
tific study of how we understand the mental states of 
others is more recent. As yet, psychologists and neu¬ 
roscientists have not agreed on common definitions of 
the terms that are used to describe and explain social 
cognition. So a few brief definitions are in order before 
we begin our study of social cognition. 


Cognition, Brain , and Consciousness, edited by B. J. Baars and N. M. Gage 
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TABLE 14.1 Perspective in social cognition and in science 



First-person perspective 

Second-person perspective 

Third-person perspective 

Data type 

Phenomenological data 

Empathic understandings 

Ordinary empirical data; 




sense data 

Methods of data gathering 

Controlled introspection on internal 

Reflection on empathic attunement; internal scanning 

Objective observation of 


states; traditional psychophysics 

of emotional and cognitive responses in the presence 
of another 

external objects 

Linguistic markers 

I feel, I know 

You seem. You look 

It is. S/he is 


1.1 Terms that are used to refer to social 
cognition 

In the research literature, terms that refer to aspects of 
social cognition are often used interchangeably and in 
different ways by different researchers. 

Empathy carries the sense of feeling the feelings of 
others. In Latin, the word means 'feeling inside' or 
'feeling with'. On the other hand, theory of mind (TOM, 
see Section 1.3) is often used to highlight the idea that 
we normally have complex metacognitive understand¬ 
ings of our own minds as well as the minds of others - 
including cognitive and affective aspects. Similarly, 
Frith and Frith (1999) introduced the term mental- 
izing to capture the idea that, when we have a well- 
developed theory of mind, we understand ourselves 
and others not just as sensory objects but also as sub¬ 
jective beings with mental states. We understand oth¬ 
ers as having mental states that we can anticipate and 
make use of to guide our own behaviors. Mind-read¬ 
ing , like mentalizing, identifies our ability to attune 
our own behaviors to the minds and anticipated 
actions of others. 

One of the most difficult aspects of understanding 
the concept of theory of mind is understanding the 
difference between seeing others as sensory objects 
versus seeing others as subjective beings with minds 
and mental states. Having a complete TOM gives us 
the ability to go beyond the sensory into the men¬ 
tal. We can do things that those with deficient TOMs 
cannot do. Once we have a TOM we can pretend, lie, 
deceive, guess, play hide-and-seek, and predict and 
understand the full range of human emotion. People 
who have deficits in TOM (people with autism, 
e.g.) have limited abilities to do these things, as we 
will see. 

Philosophers use the term intentionality when they 
want to speak about how minds and mental states are 
always 'about something else' in a way that other physi¬ 
cal objects, i.e. body parts, are not. Our thoughts always 
have an object. For example, we think 'about' the chair. 


the book, or the idea in a way that our stomach, arm, 
or tooth are not about anything other than themselves. 
Minds have mental states; minds represent objects and 
events outside themselves. It is not clear that other spe¬ 
cies comprehend the intentional nature of minds in 
their conspecifics. Humans seem to have an implicit 
understanding of the contents of others' minds. 

A separate concept is the psychological term 'inten¬ 
tion', our ability to form an image of a goal state 
and to organize action in pursuit of that goal state. 
Be careful not to confuse these two very similar terms! 
Theory of mind abilities allow us to read the inten¬ 
tions of others and to share attention with others 
about a common focus. 

Finally, the term inter subjectivity emphasizes our 
ability to coordinate mutual interactions in light of 
our perception of the subjectivity and intentionality of 
others. When this ability is absent, we readily recog¬ 
nize the deficiency in the social exchanges of others. 
Examples are found in autistic spectrum disorders, in 
the sometimes deficient emotion recognition of schizo¬ 
phrenia, and in the empathic failures of psychopathic 
and borderline personalities. 

1.2 The importance of perspective: the first, 
second, and third person 

Science usually works from a third person perspective. 
This means that researchers adopt an objective point 
of view, seeing all evidence as a physical object. Even 
human beings are seen as objects, as sensory surfaces. 
The mental 'insides' of human beings are also viewed 
through objective means such as behavioral observa¬ 
tion or brain imagery. Recently, scientists interested 
in consciousness have begun arguing for an addi¬ 
tional way of conducting science that appreciates and 
accepts data gathered from a first person perspective, i.e. 
using phenomenological data from introspection or 
self-report. To a certain extent, self-report under con¬ 
trolled circumstances is a well-established scientific 
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method, for example, in psychophysics and the study 
of perception. 

What about the second person perspective ? In this 
stance, the other person is viewed as a subject rather 
than an object, as someone who has mental states. This 
perspective is less well established in psychological 
science and neuroscience, although it is well known in 
philosophy and in clinical psychology. Contemporary 
social cognition research comes close to examining the 
second person perspective. To be exact, we are adopt¬ 
ing a third person perspective (objective view) on other 
people as they engage in a second person activity. 

1.3 Approaches to perceiving others’ minds 

Once past our fourth birthday (whether we are nor¬ 
mally developing or developmentally delayed), we 
human beings give indications of understanding other 
minds - 'mentalizing' as Chris Frith has called it. We 
can recognize and respond to the invisible, internal 


subjective regularities that account for the behaviors 
of others. We will call the full-fledged ability to under¬ 
stand and predict our own and others' minds theory of 
mind (TOM). TOM has been explained by three kinds 
of theories: module theories, theory theories, and sim¬ 
ulation theories. 

According to module theories, such as that of Simon 
Baron-Cohen, human beings develop a theory of mind 
module (TOMM) that is separate from but builds on 
other mental abilities that may be shared with non¬ 
human primates and other mammals; only humans 
are presumed to have a complete TOMM. This kind of 
theory fits well with findings from the study of autism. 

Theory theories suppose that TOM capabilities 
develop as a primitive, implicit theory over the course 
of development, much like Piaget's conservation theo¬ 
ries. Such implicit theories predict abrupt changes in 
behavior as new knowledge is added, as is seen in the 
abrupt change in children's understanding of their 
own minds between ages three and four. 


FRONTIERS OF COGNITIVE NEUROSCIENCE 


The Social Brain 


■ 




FIGURE 14.1 Christopher Frith, PhD, Wellcome Trust 
Centre for Neuroimaging, University College London, UK. 


The tradition in cognitive neuroscience has been to 
study people and their brains in isolation, but the past 
few years have seen a dramatic increase of interest in 
what happens when people interact with one another. 
Major buzz-words include theory of mind: our ability to 
read the minds of others, and mirror neurons : neurons 
that respond when we act and also when we see some¬ 
one else act. For me, the most exciting result from these 
new studies is the demonstration of how embedded we 
all are in the social world. We may feel like independent 
agents, but, in fact, we are constantly being buffeted one 
way and another by a stream of social signals to which 
we respond whether we like it or not. Mostly we are not 
aware that we are responding to these signals. 

This idea is confirmed by many experiments, but at 
the moment I have two favorites that demonstrate such 
automatic responses. The first (Blakemore et al, 2005) 
used fMRI to see what happens in the brain when we see 
someone else being touched. Volunteers were scanned 
while they saw a video of someone else being touched 
on the face and also when they were touched on the 
same spot themselves. The scanning data (Figure 14.2) 
clearly shows that same region of somatosensory (body 
touch) cortex was activated whether the participants 
were touched or whether they saw someone else being 
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FIGURE 14.2 These brain regions are active when we feel 
our face being touched and when we see someone else being 
touched on the face. Source : Blakemore et ah, 2005. 


touched. The location of the activity depended on the 
precise location of the touch: face or neck, left or right 
side. The effect did not occur if the participants saw 
some inanimate object, like a somewhat face-shaped 
electric fan, being touched. In the vast majority of cases 
people are not aware that they are feeling someone else 
being touched. Only in rare cases of synesthesia does the 
activity break through into consciousness, resulting in 
reports that 'when I see someone's face being touched I 
feel it on my own face.' 

The second experiment (Liepelt et ah, 2009) used the 
classic psychologist's measure of reaction time. In this 
study participants had to press buttons with their first 
or second finger as quickly as possible. The signal to 
perform this action appeared in a picture of a hand 
(Figure 14.3). A 1 or a 2 appearing between the first 
and second fingers of this hand indicated which but¬ 
ton to press. The clever feature of this experiment was 
that, on some occasions, the first and second fingers 
of the hand in the picture were held down with metal 
clamps. On these occasions the reaction times of the 
participants were significantly slower, even though 
their own hands were completely free at all times. Here 



FIGURE 14.3 Experimental design and results for finger responses. Source : Liepelt et ah, 2009. 
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again the effect was very specific. Only the fingers that 
are constrained in the picture were slowed down in the 
participants. 

We are not aware of it, but the mere sight of what is 
happening to the people around us alters our own feel¬ 
ings and behavior. Perhaps our brain is essentially a 
social organ. 
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Simulation theories suppose that we understand 
other minds by internally simulating or 'running off 
line' the mental states of others in each situation. The 
dual responsiveness of mirror neurons to self- and 
other-generated action could be taken as support for 
simulation theory. 

It seems very likely that all three kinds of theories 
are needed to account for human 'mentalizing' abilities. 
As we will see, there are separable skills that develop in 
mammals and humans that operate much like modules ; 
we can lose one module but still have the other. The 
system that allows us to imitate others seems to oper¬ 
ate through internal simulation of the actions of others. 
Finally, adult human beings have sophisticated social 
perception abilities that allow us to reason about other 
people's internal states; we act as if we have a complex 
set of rules about our own and others' mental states 
that could be called an implicit theory. 

2.0 AN ORGANIZING FRAMEWORK FOR 
SOCIAL COGNITION 

Simon Baron-Cohen (1995) hypothesized that a fully 
developed theory of mind is composed of four kinds 
of skills that develop independently. These skills are 
detection of intentions of others , detection of eye-direction , 
shared attention , and a complex repertoire of implicit 
knowledge about others, which he called the theory 
of mind module. Some of these skills are observed in 
mammals and non-human primates as well as in 
humans. However, only the theory of mind module 
appears in normally developing human beings. We 
will first introduce Baron-Cohen's model (summa¬ 
rized in Figure 14.4) and then use it as a way to orga¬ 
nize the larger body of social cognition research. 

2 .1 Intention 

The first component of Baron-Cohen's TOM is called 
the intentionality detector (ID). This is the ability to 


perceive intention or purposeful action in many forms 
of biological and non-biological movement. For exam¬ 
ple, when we watch leaves swirling in a parking lot, 
we have a tendency to see the leaves as 'wanting to 
go together'. We ascribe common purpose to the pile 
of leaves. Or, when we watch pieces of modeling clay 
being moved around an artificial landscape in clay- 
mation films, we readily attribute intentions and other 
mental states to the pieces of clay. Likewise, when we 
watch people and animals engaged in behaviors, we 
seem to understand their goals and the desired out¬ 
comes of their actions. We interpret action as intention. 

2*2 Eye detection 

The second component of the model is the eye direction 
detector (EDD), the skill to detect eyes and eye-like stimuli 
and to determine the direction of gaze. Many mammals 
seem to have the ability to notice and use information 
about eye direction. Cats, for example, use eye direction 
as part of their social dominance behavior with other 
cats; the non-dominant cats must avert their eyes in the 
face of the dominant cat. Humans, from the first hours of 
life, search for and focus on the eyes of their caregivers. 
We also have a strong tendency to see non-living stim¬ 
uli as eye-like; hence, we see the 'man in the moon' and 
faces on automobiles, gnarled trees, and mountains. The 
'language of the eyes' seems to be a fundamental means 
of communicating mental states among humans. 

Both the intentionality detector and the eye direction 
detector involve dyadic (two-way) interactions. That is, 
there is one perceiver and one object of perception. As yet, 
no sharing of mental states is necessarily involved. 
Both EDD and ID are found in non-humans as part of 
their social perception abilities. It is the third module of 
TOM that is unique to human social cognition. 

23 Shared attention 

The shared attention mechanism (SAM) is the ability we 
have, by the end of our first year of life, to understand 
that when someone else shifts his or her direction of gaze 
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Baron-Cohen’s theory of mind model 


Stimuli with self-propulsion & direction: 
Intentionality Detector (ID) 


i 


Eye-like stimuli: 

Eye-Direction Detector (EOD) 


1 



Triadic representations 



Knowledge of mental states used to 
understand and predict self and other: 
Theory of Mind Module (TOMM) 


FIGURE 14.4 A schematic dia¬ 
gram of Baron-Cohen's theory of mind 
model with the eye-direction detector 
(EDD), shown on the upper right, sending 
inputs to the intentionality detector (ID), 
upper left, and to the shared attention 
mechanism (SAM), shown in the center 
of the diagram. The SAM also receives 
inputs from the ID and interacts with the 
theory of mind module (TOMM). Source : 
Adapted from Baron-Cohen, 1995. 


he or she is looking at' something. We seem to learn that 
looking leads to seeing - an advance over the simpler 
signal of eye direction. We realize that we can look too 
and see the same thing. Gaze shifting and social point¬ 
ing of fingers are ways we learn to direct the attention of 
a companion. 

Infants before 1 year of age, most other primates, 
and other mammals do not have a shared attention 
ability. We can see this in our much loved companion 
animals. While our family dog may chase a ball and 
bring it back, he will not follow our gaze if we look 
toward a ball lying in the grass. He will not follow our 
pointing finger when we try to direct his gaze toward 
the ball. The dog has considerable intelligence but 
does not have shared attention. Similarly, an infant at 
6 months does not turn her head to follow the caregiv¬ 
er's gaze; a 1-year-old does. Shared attention abilities 
mark the human species. 

2*4 Higher-order theory of mind 

The final component of full-fledged theory of mind is 
what Baron-Cohen has called the theory of mind module 
(TOMM), a complex knowledge base containing rules 
of social cognition that develops by the time we reach 
our fourth birthday. TOMM tells us that: 

• Appearance and reality are not necessarily the 
same - a rock can look like an egg but not be an 
egg; I can pretend to be a dog but not be a dog 


• A person who is sitting still in a chair may be 'doing 
something', i.e. thinking, imagining, or remembering 
(young children do not appreciate this) 

• Other people can have mental states as well as 
physical states 

• Other people can know things that I don't know; I 
can be fooled or deceived; I can detect deception 

• I can know things that other people don't know: I 
can fool or deceive others; I understand the point 
of games like hide-and-seek 

• My mental state in the past was different from how 
it is now 

• Facial expressions are indicators of mental states as 
much as they are indicators of physical states; I can 
distinguish a surprised face from a yawning one. 

TOM is not the same as intelligence or IQ. Develop- 
mentally delayed children and adults display complete 
TOM abilities despite low IQs, while people living with 
autistic spectrum disorders (ASD) may have high IQs 
but markedly deficient TOM abilities. 

We can now use Baron-Cohen's four TOM skills as a 
way to organize and guide our study of social cognition. 

3.0 MIRROR NEURONS AND 
INTENTION DETECTION 

Where is social cognition located in the brain? Some 
recent investigations of mirror neuron systems in the 
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macaque monkey have brought to light some dramatic 
findings regarding where social and intentionality sys¬ 
tems may be processed in the brain. Initial studies using 
neuroimaging techniques such as fMRI to investigate a 
possible similar system in humans provided intrigu¬ 
ing new evidence that such a system may exist. More 
recent studies, however, cast a shadow on earlier find¬ 
ings. Existence of a human mirror neuron system is still 
an open and hotly debated topic in the field of social 
cognitive neuroscience. We begin our discussion with 
findings for the macaque monkey and then review the 
evidence for mirror neurons in a human brain. 

3*1 From action to intention 

The mirror neuron system (MNS) is a collection of cor¬ 
tical neurons that is theorized to allow one to under¬ 
stand the intentions of others from observation of their 
actions. Mirror neurons were first discovered in the 
frontal cortex of macaque monkeys and shortly thereaf¬ 
ter in their parietal cortex. These cortical neurons have 
the remarkable property that the individual mirror neu¬ 
ron fires not only when a particular action is perceived 
but also when the observer performs the same action. 
Immediately, researchers grasped the possibility that 
mirror neurons might be a means of comparing one's 
own actions with those of others. In addition, mirror 
neurons are being examined as a basis for inferring the 
goals and intentions of others through internal match¬ 
ing of action representations of others' actions with 
action representations in one's own action repertoire. 

The original work on mirror neurons was con¬ 
ducted by Rizzolatti et al. at the University of Parma 
in Italy in the early 1990s. They found mirror neu¬ 
rons in the frontal motor area of monkey cortex in 
an area labeled F5. Using single-cell recordings of 
macaque cortical neurons, they found that such neu¬ 
rons would respond when the experimenter grasped 
peanuts placed on a board as well as when the mon¬ 
key grasped the peanuts; the neuron did not respond 
when the peanut was observed alone on the board 
or when the experimenter grasped the peanut with a 
tool. Clearly, the act of grasping food with one's hand 
was the act that the neuron was 'tuned' to. Figure 14.5 
illustrates a monkey with a microelectrode apparatus 
affixed to his head; he is sitting in a functional mag¬ 
netic resonance imaging (fMRI) recording apparatus. 
Figure 14.6 shows the experimental arrangement for 
assessing responses of single mirror neurons in differ¬ 
ent conditions used by Rizzolatti and his colleagues. 

To insure that the single-cell recordings were not 
an artifact of monkey-experimenter interaction or 



FIGURE 14.5 Monkey with a microelectrode attached to his 
head, performing inside an fMRI apparatus, a, microelectrode; b, 
fMRI coil; c, cradle. Source : Tammer et al., 2006. 


food expectancy, the researchers also recorded from 
the monkey while it observed nearby food grasp¬ 
ing actions between the experimenter and a second 
monkey. The mirror neuron responded as before, i.e. 
it responded when the observed monkey grasped the 
food, not when the food was moved, but again when 
the observed monkey grasped the food (Figure 14.7). 

Research with macaque monkeys has been done 
with single-cell recordings of individual prefrontal 
neurons. The existence of mirror neurons in macaques 
is well established. Similar single-cell recordings of 
neurons have not been conducted in humans; evidence 
for the human mirror neuron system (MNS) comes 
from imaging data that aggregates the activity of many 
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FIGURE 14.6 The observed acts and single-cell responses of 
mirror neurons, (a) The experimenter places a piece of food on the 
board, moves it toward the monkey, and the monkey grasps the 
food. The figure at the top of the panel illustrates the acts. Individual 
responses of the neuron over time are presented in the middle of the 
panel. At the bottom is a histogram representing the total responses 
in each 20-millisecond time segment over time. Notice the numer¬ 
ous responses when the experimenter grasps the food, the lack of 
responses while the board is moved, and the numerous responses 
again when the monkey grasps the food, (b) The experimenter grasps 
the food with a tool, moves the food toward the monkey, and the 
monkey grasps the food with its hand. Notice here that responses 
occur only when the monkey's hand grasps the food, (c) The monkey 
grasps the piece of food in the dark. Notice the numerous responses 
to the grasping act even when it is conducted in the dark. Source : 
Rizzolatti et al., 1996. 
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FIGURE 14.7 Observed acts of another monkey can evoke 
mirror neuron responses. The mirror neuron responds to observed 
action of another monkey (a), of the experimenter (b), and of the 
recorded monkey itself (c). Source : Rizzolatti et al., 1996. 


neurons. Thus, individual human mirror neurons 
have not been studied to date. Nevertheless, there are 
marked similarities between mirror neuron systems 
studied in monkeys using single unit recordings and in 
human using neuroimaging of population-level neuro¬ 
nal responses. Rizzolatti et al. (2002) have summarized 
the monkey to human homologies in Figure 14.8. 

Area F5 in the macaque (purple in Figure 14.8) and 
Brodmann area 44 in humans (pink) seem to code for 
hand actions, such as those studied in the original mir¬ 
ror neuron research. Area F4 in the macaque and the 
ventral premotor cortex (vPM or lower Brodmann area 
6 in humans (colored red in Figure 14.8) both respond 
to arm and wrist movement. Individual frontal neu¬ 
rons in the macaque and prefrontal areas in humans 
are tuned to arm and wrist movements toward partic¬ 
ular locations in the space around the individual. This 
implies that mirror neurons in these areas are not sim¬ 
ply representing general movements of the arm and 
wrist, but rather they are responding to acts connected 
to particular goals, i.e. reaching locations in space. 

Note: you may remember that Brodmann area 44 
in the left hemisphere has been called Broca's area. 
It has traditionally been seen as the cortical speech 
area. Recent studies of imitation in humans have 
shown that Brodmann area 44 has mirror neuron 
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(b) 

FIGURE 14.8 Monkey to human homologies in mirror neu¬ 
ron locations and functions, (a) A lateral view of the macaque right 
cortex, (b) A lateral view of the human right cortex. Source : Rizzolatti 
et ah, 2002. 


capabilities, involving representation of actions with 
the hand and arm (Iacoboni, 2005; Figure 14.9). Thus, 
Broca's area has mixed abilities. 

A long-standing challenge in the study of mir¬ 
ror neurons has been distinguishing between simple 
action recognition (is that grasping, reaching, or hold¬ 
ing?) and intention detection (where an agent has a goal 
which is accomplished by the action: drinking tea 
versus cleaning up after tea). The final piece of evi¬ 
dence linking mirror neurons to intention detection 
comes from studies of human acts carried out in dif¬ 
ferent contexts. A central question in these studies is: 
does the MNS respond to the act regardless of context 
or does it respond to acts in particular contexts? The 
first alternative implies that mirror neurons do action 
recognition. The second allows us to extend the infer¬ 
ence: MNS is for intention detection. 

Marco Iacoboni and his colleagues have obtained 
evidence supporting the argument that the MNS 
responds to intentions rather than particular actions 



\ V 


FIGURE 14.9 A simplified view of the left hemisphere with 
locations of the theorized frontal mirror neuron system highlighted. 
(Note: Inferior parietal cortex = supramarginal gyrus = Brodmann 
area 40; ventral premotor cortex = lower Brodmann area 6; poste¬ 
rior inferior frontal gyrus = pars opercularis of IFG = Brodmann 
area 44.) 


Context Action Intention 



After tea Cleaning up 


FIGURE 14.10 Two different contexts, acts, and intentions used 
in Iacoboni et at. (2005): drinking tea versus cleaning up after tea. Left 
panel shows the stimuli in the Context condition, center panel shows 
the Action condition, and right panel shows the Intention condition. 
Source : Iacoboni et al, 2005. 


by examining fMRI images as participants observe 
acts with and without context. They found that actions 
embedded in contexts (Intention in Figure 14.10), 
compared with the other two conditions (Context only 
or Action), yielded a significant fMRI increase in the 
posterior part of the inferior frontal gyrus and the 
adjacent sector of the ventral premotor cortex where 
hand actions are represented. In order to demon¬ 
strate that intention could be assessed separately from 
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action, the researchers compared fMRI images from 
the two intention and action conditions. These com¬ 
parisons are shown in Figure 14.11. 

Follow-up studies by Dinstein and colleagues (2007, 
2008) and Caramazza and colleagues (2009), however, 
have not supported these earlier findings. Dinstein 
and colleagues (2008) found that while areas in the 
parietal lobe (the anterior intraparietal sulcus) were 
active for perception and execution of movements, 
the pattern of activity differed sharply depending on 
whether the movement was observed or executed (see 
Figure 14.12 for an illustration of their experimental 
design). Dinstein and colleagues suggested that these 
results indicate a differing subpopulation of neurons 
were supporting the observation versus execution of 
movements (Figure 14.13). 



FIGURE 14.11 fMRI results from Iacoboni et ah (2005). Upper 
panel shows response in the Intention minus Action conditions; lower 
panel shows Intention minus Context conditions. Source : Iacoboni 
et ah, 2005. 


A related study by Caramazza and colleagues 
(2009) provided related evidence in an fMRI study 
investigating adaption of neurons within brain areas 
suggested to hold mirror neurons. They found that 
these brain areas adapted when hand gestures were 
observed, but not when they were executed. 



FIGURE 14.12 The experimental paradigm used in Dinstein 
et ah, 2008. The subjects played a version of "rock, paper, scissors' in 
the scanner. When the subject sees the word 'GO' at the beginning 
of the trial, he executes a game movement (hand gesture for 'rock', 
'paper', or 'scissors'). After a brief (3-6 s) delay, he then observes his 
opponent's movement. This was repeated for 36 executions and 36 
observations. Source : Dinstein et ah, 2008. 
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Yellow: rock biased voxels 
Green; paper biased voxels 
Purple; scissors biased voxels 
Blue: inconsistently biased voxels 


FIGURE 14.13 Brain activity for executed 
versus observed movements. The results from 
the Dinstein et ah, 2008 'rock', 'paper', 'scissors' 
study show that differing brain areas are acti¬ 
vated for executing versus observing closely 
matched hand movements. These findings argue 
against some theories of the human mirror neu¬ 
ron system. 
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TABLE 14.2 Ethological review of eye/gaze processing across classes of animals 


Subject group 

Eye presence 

Simple gaze 

Gaze following 

Joint attention 

Mental attribution 

Fish 

/ 

? 

? 

? 

? 

Reptiles 

/ 

/ 

? 

? 

? 

Birds 

/ 

/ 

? 

/(?) 

? 

Rodents 

/ 

? 

? 

? 

? 

Dogs (domestic) 

? 

/ 

/ 

? 

? 

Prosimians 

? 

/ 

X 

? 

? 

Monkeys 

/ 

/ 

// 

/ 

X 

Great apes 

Human 

/ 

/ 

/ 

/ 

X(?) 

- 6 months 

/ 

X 

X 

X 

X 

- 9 months 

/ 

/ 

X 

X 

X 

-12 months 

/ 

/ 

X 

/ 

X 

-18 months 

/ 

/ 

/ 

/ 

X 

- 24 months 

/ 

/ 

/ 

/ 

X 

- 48 months 

/ 

/ 

/ 

/ 

/ 

- Autism 

/(?) 

/ 

? 

X 

X 

- Down syndrome 

/ 

/ 

/ 

/ 

/ 

- Schizophrenia 

/ 

/ 

? 

? 

? 

- Amygdala damage 

/(?) 

X 

X(?) 

? 

X(?) 


(from Emery, 2000) 

/ = positive evidence; X = no evidence; ? = not tested or controversial evidence 


Cumulatively, these and other recent studies of 
the human MNS are providing evidence that the pro¬ 
cesses of observing and executing a motor act do not 
bear as many similarities across human and macaque 
brains as initially thought. Studies of the MNS are a 
good example of how scientific theories develop and 
change through ongoing experimentation, with new 
and revised hypotheses tested as the experiments and 
the theory are refined and improved. 


3.2 Eye detection and gaze perception 

Perception of eyes in conspecifics and guiding of social 
behavior in light of that perception occurs in many 
classes of animals from reptiles to humans. A clear 
understanding of the role of the eyes in social cognition 
requires that we understand the various kinds of eye and 
gaze processing that have been studied. In a review of lit¬ 
erature including numerous animal classes, Emery (2000) 
showed an evolutionary trend in complexity of gaze 
processing. Table 14.2 summarizes Emery's findings. 

While fish, reptiles, birds, and other mammals 
have some ability to process eye-like stimuli (horizon¬ 
tal pair of dark circles) and perceive gaze direction, 
only great apes (gorillas, chimpanzees, bonobos, and 
orang-utans) and humans have shared attention and 
can use the 'language of the eyes' to understand men¬ 
tal states of others. 

Several types of eye and gaze processes are shown 
in Figure 14.14. Where do these processes take place in 



(1) Mutual versus (2) Gaze following (3) Joint attention 
averted gaze 



^ 0 ® 




^ 00 ^ 

(4) Shared attention 

FIGURE 14*14 Eye and gaze processing. (1) Mutual gaze is 
where the attention of individuals A and B is directed to one another. 
Averted gaze is when individual A is looking at B, but the focus of 
B's attention is elsewhere. (2) Gaze following is where individual A 
detects that B's gaze is not directed toward them and follows the light 
of sight of B to a point in space. (3) Joint attention is the same as gaze 
following except that there is a focus of attention (an object) so that 
individuals A and B are looking at the same object. (4) Shared atten¬ 
tion is a combination of mutual attention and joint attention, where 
the focus of attention of A and B is on an object of joint focus and on 
each other (T know you're looking at X and you know that I'm looking 
at X'). (5) Theory of Mind relies on 1-4 as well as higher order social 
knowledge that allows individuals to know that the other is attend¬ 
ing to an object because they intend to do something with the object. 
Source : Emery, 2000. 
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FIGURE 14.15 Superior temporal sulcus and intraparietal sulcus: 
eye and gaze detection areas. 

the brain? The superior temporal sulcus registers eyes 
and eye-like stimuli. The more complex levels of gaze 
processing (gaze direction and detection of gaze aver¬ 
sion) involve connections between STS and areas in the 
parietal lobe, particularly the intraparietal sulcus (IPG; 
Haxby et al. 2000) (Figure 14.15). In addition, connec¬ 
tions of STS and IPG with subcortical structures, such 
as the amygdala, allow us to register the social and 
emotional significances of gaze, including threat. 

Social information from eyes and gaze direction 
come from the changeable aspects of the human face. 
We can also use visual information to detect the invari¬ 
ant aspects of individual faces , such as identity. These 
aspects of face perception occur in a separate area of 
the temporal cortex discussed in a section later in this 
chapter. 

Finally, attribution of mental states involves con¬ 
nections of the STS to the MNS and to medial frontal 
cortical areas (Iacoboni, 2005). We will discuss attribu¬ 
tion of mental states in greater detail shortly. 

3*3 Shared attention 

Shared attention seems to be a social skill that is 
unique to great apes and humans. Remember shared 
attention is more than simply looking at the same 
thing that another person is looking at. Shared atten¬ 
tion involves the additional qualification that the two 
observers not only observe the same object but also 
know that the other is looking at the object. It is a 
triadic (three-way) activity. Shared attention allows 
us implicitly to recognize that: 7 know that you are 


BOX 14.1 Gaze-following experiment 


Try this experiment with shared attention: when you 
are in a gathering of friends or classmates, shift your 
gaze to the corner of the room without saying any¬ 
thing. See whether others follow your gaze. Ask them 
why they did what they did. 



FIGURE 14.16 An 18-month-old pointing. Pointing is a sign 
of triadic interaction. Source: Brooks and Meltzoff, 2003. 


looking at that'. Apes and humans seem to know that 
when conspecifics are gazing at something, they are 
also internally representing it. Looking leads to see¬ 
ing. If I want to see what you see, I can follow your 
gaze (see Box 14.1). 

How do these shared processes get set up in 
humans? Brooks and Meltzoff (2003) have shown that 
human infants begin to follow the direction in which 
an adult turns his or her head by the age of 9 months; 
however, at 9 months, the infant follows head direction 
whether the model's eyes are open or closed. By 12 
months of age, the infant will follow gaze more often 
when the model's eyes are open that when they are 
shut. The infant follows gaze rather than head direc¬ 
tion now. Shared attention has developed. The infant 
seems to know implicitly that open eyes allow look¬ 
ing; looking leads to seeing (Figure 14.16). 

What areas of the brain support shared attention? 
We know that STS supports eye detection. In order to 
move from simple detection to shared attention, areas 
in the prefrontal cortex become involved. Williams 
and colleagues (2005) studied adults when they were 
experiencing shared attention compared to a control 
condition (Figure 14.17). 
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FIGURE 14.17 Stimuli used to create joint attention. When we look at the red dot on the left, we have the 
sense that the man is looking at the same object as we are; looking at the red dot on the right does not lead to the 
same sense of shared attention.Sowrce: Williams et ah, 2005. 
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FIGURE 14.18 Networks for shared attention. 


Images from fMRI analyses show that shared atten¬ 
tion recruits frontal areas including the ventromedial 
prefrontal cortex (VM-PFC), the left superior frontal 
gyrus (BA 10), cingulate gyrus, and caudate nucleus. 
The VM-PDF is associated with registering the men¬ 
tal state of the other as we will discuss next. Williams 
and colleagues speculate that BA 10 is responsible for 
matching perception and action (Figure 14.18). 


3 A Higher-order TOM abilities 

3 . 4.1 Attribution of mental states to ourselves 
and others 

In the previous sections, we found that the paracingulate 
is involved in imitation learning and that the ventrome¬ 
dial prefrontal cortex (VM-PFC) is involved in shared 
attention. In this section, we will examine the role that 
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FIGURE 14*19 Divisions of prefrontal cortex: DM-PFC, M- 
PFC, and VM-PFC. Superior frontal gyrus, cingulate (BA 24), and 
paracingulate (BA 32) are areas shown to be important in attribu¬ 
tion of mental states. 


the medial wall of the PFC plays in attribution of mental 
states to others, or, as Chris Frith calls it, / mentalizing\ 
The medial wall of prefrontal cortex can be divided 
into three segments from top to bottom: dorsomedial 
prefrontal cortex (DM-PFC), medial prefrontal cortex 
(M-PFC), and ventromedial prefrontal cortex (VM-PFC). 
(They are divided by convention according to their 
Talairach coordinates in three-dimensional space, which 
we will not worry about here.) DM-PFC includes the 
cortex at the top of the medial wall of the prefrontal cor¬ 
tex. M-PFC is the middle section of the medial wall of 
PFC. VM-PFC is composed of the bottom of the prefron¬ 
tal lobe and the lower inside wall of the prefrontal cor¬ 
tex (Figure 14.19). 
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FIGURE 14*20 Anatomy of cingulate and paracingulate gyri: the upper left panel shows the left hemisphere, and the upper right panel 
shows the right hemisphere. The paracingulate sulcus is shown in blue, the cingulate sulcus is shown in pink, and the callosal sulcus is shown 
in purple. Lower panel shows these regions in more detail, including the rostral portion of the cingulate zone. Source : Heckers et al., 2004. 


The cingulate and paracingulate gyri, which are 
sometimes spoken of collectively as the anterior cingu¬ 
late cortex (ACC), form a belt (cingulum in Latin) around 
the corpus callosum. In addition, depending on how the 
gyri of individual brains are folded, the paracingulate 
gyrus may be folded into a sulcus. Figure 14.20 helps us 
see the anatomy of the cingulate and paracingulate gyri. 
Figure 14.21 depicts brain areas active for perspective 
taking and intentional stance. 

3 . 4.2 Perspective taking and intentional stance 

Perspective taking is a social skill that is fundamental 
to human empathy. It allows us to understand how 
another person thinks and feels about a painful situ¬ 
ation. For example, when people are asked to imag¬ 
ine the pain that they or another person would feel 
(compared to how an artificial limb would feel) when 
their fingers are pinched in a car door, the medial 


prefrontal cortex is significantly activated as well as 
other cortical areas associated with pain perception 
(Jackson et al., 2006). Specifically, the paracingulate 
(BA 32) cortex of the M-PFC is significantly activated 
in both perspectives; it registers imagined self and 
other pain. In addition, when participants were imag¬ 
ining their own pain, the cingulate cortex (BA 24) was 
also active. This difference reflects our ability to empa¬ 
thize with the pain of others but also to distinguish 
our own pain from theirs. 

Or consider playing a game with another: gam¬ 
ers and athletes speak of 'psyching out' their oppo¬ 
nent. A lot of what they mean is that they read the 
other person's mind. To do this, we put ourselves in 
the other person's shoes or, to say it another way, we 
use empathy. When we play the game 'Rock, scissors, 
paper' against an unseen person (compared to an 
unseen computer), we adopt an intentional stance - 
we act as if the opponent is another subjective being. 
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(b) 

FIGURE 14*21 Perspective-taking and intentional stance, (a) 
Significant medial prefrontal activation for self versus artificial pain, 
(b) Significant M-PFC activation for other versus artificial pain, (c) 
Change in regional blood flow in the PET procedure in M-PFC; 
self = blue line; other = yellow; artificial limb = red. Source : Jackson 
et al., 2006. 



FIGURE 14*22 Activation in anterior paracingulate cortex 
when we mentalize about our opponent. Source : Gallagher et al., 2002. 


And, we can see a difference in the metabolic activ¬ 
ity of the brain that coincides with our adoption of 
an intentional stance. In Figure 14.22, we can see that 
the anterior paracingulate cortex (BA 32) is activated - 
that part of the medial prefrontal cortex implicated 
in perception of mental states in ourselves and others 
(Gallagher et al., 2002). 



FIGURE 14*23 Adopting the perspective of another to make 
health care decisions for him or her. Medial PFC activation appears 
in yellow toward the right. Source : Ruby and Decety, 2003. 



FIGURE 14*24 Thinking about social relationships. Dorsomedial 
prefrontal cortex and medial parietal cortex system for thinking about 
social relationships. Source: Iacoboni et al., 2004. 


We are sometimes asked to make decisions for 
others, keeping in mind what they would want. In a 
study where medical students were asked to adopt 
the perspective of a patient in order to make puta¬ 
tive health care decisions for them, PET scans again 
reveal unique medial frontal activity, in this case, in 
the medial superior frontal gyrus (Figure 14.23) (Ruby 
and Decety, 2003). 

PET activation appears in the dorsomedial prefrontal 
cortex (DM-PFC) and medial parietal (precuneus) when 
study participants watch a film of two people engaged 
in social interaction (Figure 14.24) (Iacoboni et al., 2004). 
The same areas are not active when participants watch 
a single person engaged in solitary activity. The red 
areas in the figure show activity present during social 
interaction but not solitary action. The blues areas show 
activity that was present while participants watched 
social interaction compared to resting state. 
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Iacoboni and his colleagues (2004) suggest that: 

The dorsomedial prefrontal cortex and medial parietal 
cortex system for thinking about social relationships 
is apparently part of the brain's default state circuitry; 
it may continuously, often without effort or intention, 
assess and analyze past, present, or possible future 
social relationships whenever non-social tasks do not 
demand full attention (p. 1171). 

3*5 Social cognition of others like and unlike 
us: I-It in the brain? 

Of significance for social psychology is the question 
of how the brain is involved when we perceive oth¬ 
ers who are like us versus others who are unlike us. 
When other people are not perceived as belonging to 
our social in-group, we may feel justified in treating 
them differently. Among other things, we may deny 
that they experience higher mental states and feelings. 
In addition, dissimilar 'others' are more likely to be 
treated as objects in Buber's (Figure 14.25) sense of I- 
It (see Box 14.2). It appears that we are close to being 
able to see brain activity that corresponds to Buber's 
distinction. 

Research by Mitchell et al. (2006) reveals distinct acti¬ 
vation differences in the medial prefrontal cortex of par¬ 
ticipants who are asked to make inferences about the 
preferences of other people who are like them (similar) 
or not like them (dissimilar) politically. They found that 


areas in the ventromedial prefrontal cortex (VM-PFC) are 
active during judgments about similar people, whereas 
areas in the dorsomedial PFC were active in judgments 
about dissimilar individuals (Figure 14.26). 

Mitchell and his colleagues conclude that, based on 
their results, we may actively deploy a different set of 
social cognition strategies in perceiving dissimilar oth¬ 
ers. Using less of the VM-PFC's perception of dissimi¬ 
lar others implies attributing less feeling and emotion 
to them. The next section addresses the issue of differ¬ 
ent strategies for perception of others. 

3 . 5.1 Cognitive versus affective empathy? 

Psychologists sometimes make a distinction between 
cognitive empathy and affective empathy. They point 
out a difference between theory of mind skills that 
draw on understanding of others' beliefs and empathic 
skills that help us understand how another person 
feels. 

Shamay-Tsoory et al. (2005) found differences 
reflecting this when they tested adults with ventro¬ 
medial PFC lesions on different kinds of social infer¬ 
ence tasks. The researchers asked their participants to 
answer questions about three kinds of stories, those 
involving (1) second-order false beliefs, (2) social 
faux pas , and (3) detection of irony. Box 14.3 summa¬ 
rizes the stories and questions that they used. The 
researchers believed that understanding social faux pas 


BOX 14.2 In 1923, philosopher Martin Buber wrote about the second person perspective in his classic book 

I and Thou 



FIGURE 14.25 Martin Buber (1878-1965). 

Buber identified two fundamentally different ways 
of being in relation to other people and objects, I-It and 
I-thou. I-It involves perceiving others as objects; I-Thou 


involves an empathic perceiving of others as subjects. 
Ordinarily we can move easily between these two ways 
of perceiving others. As we adopt one or the other stance, 
our own internal states change. Within the past decade, 
these internal states have been assessed via brain imag¬ 
ing techniques. 

Buber wrote: 

Primary words are spoken from the being. 

If Thou is said , the I of the combination I-Thou is said along 
with it. 

If It is said , the I of the combination I-It is said along with 
it. . . . 

There is no I taken in itself, but only the I of the primary 
word I-Thou and the I of the primary word I-It. .. . 

When Thou is spoken , the speaker has no thing for his object. 
For where there is a thing there is another thing. Every It is 
bounded by others; It exists only through being bounded by 
others. But when Thou is spoken , there is no thing. Thou has 
no bounds. 
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Ventral M-PFC 



(b) 


FIGURE 14*26 Understanding similar and dissimilar others, 
(a) Ventromedial PFC activation in judgments of similar others, (b) 
Dorsomedial PFC activation in judgments of dissimilar others. Source : 
Mitchell et al, 2006. 


and irony involves greater dependence on emotional 
processing, while second-order false belief tasks rely 
more heavily on cognitive processing. 

Shamay-Tsoory and his colleagues found that 
patients who had ventromedial prefrontal lesions 
performed like the healthy control participants 
and patients with posterior lesions on the second- 
order false belief task - they all performed at ceiling. 
However, the VM-PFC patients did very poorly on 
the irony and social faux pas tasks in comparison with 
other participants. The researchers interpreted these 
findings to mean that VM-PFC is essential to affective 
empathy but not to cognitive empathic skills. 


BOX 14*3 Stories testing three kinds of social 
inference 


Second-order false belief 

Hana and Benny are sitting in the office talking about 
their meeting with their boss. Benny is putting an open 
bottle of ink on his desk. As he is doing so, some ink 
spills, so he leaves the office to look for a towel to clean 
up the spilled ink. While Benny is out of the office, Hana 
moves the ink bottle to the cabinet. While Benny is out¬ 
side the office, he looks back through the keyhole and 
sees Hana moving the ink bottle. Benny enters the office. 
Following the story, four questions were asked: 

Belief question: Where will Hana think that Benny 
thinks the ink bottle is? 

Reality question (assessing story comprehension): 

Where, actually, is the ink bottle? 

Memory question: Where did Benny put the ink bottle? 
Inference question: Where would there be an ink stain? 

Irony 

A sarcastic version item: Joe came to work and, instead 
of beginning to work, he sat down to rest. His boss 
noticed his behavior and said: foe, don't work too 
hard!' 

A neutral version item: Joe came to work and imme¬ 
diately began to work. His boss noticed his behavior, 
and said: 'Joe, don't work too hard!' 

Following each story, two questions were asked: 

Factual question (assessing story comprehension): Did 
Joe work hard? 

Attitude question (assessing comprehension of the true 
meaning of the speaker): Did the manager believe 
Joe worked hard? 

Recognition of social faux pas 

Mike, a 9-year-old boy, just started at a new school. He 
was in one of the cubicles in the bathroom at school. 
Joe and Peter, two other boys at school, came in and 
were standing at the sinks talking. 

Joe said, 'You know that new guy in the class? His 
name's Mike. Doesn't he look weird? And he's so 
short!' Mike came out of the cubicle, and Joe and Peter 
saw him. Peter said, 'Oh, hi, Mike! Are you going out 
to play soccer now?' 

The participant is then asked the following 
questions: 

Detection of the faux pas question: 

Did anyone say anything they shouldn't have said? 
Who said something they shouldn't have said? 

Why shouldn't they have said it? 

Why did they say it? 

Control question (assessing story comprehension): In 
the story, where was Mike while Joe and Peter were 
talking? 

Source: (from Shamay-Tsoory et al, 2005) 
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It appears that empathic abilities follow the general 
rule that applies to other cognitive skills: facts about 
the world, beliefs, and sensory representations are 
processed by the dorsal and lateral cortices. Feelings, 
emotional values, and social significances depend on 
ventral and medial cortices that are closely intercon¬ 
nected with the subcortical limbic system. 

3.6 Face perception 

Perception of the unchanging aspects of the human 
face occurs in the fusiform face area (FFA), which is 
part of the inferior temporal lobe (Figure 14.27). 

As an example of how the FFA looks in a brain 
image, we can look at the PET/MRI image from 
Caldara and colleagues (2006) (Figure 14.28). These 
researchers compared cortical activation when par¬ 
ticipants observed objects versus human faces. In the 
image, we are looking at the bottom of the brain. The 
temporal lobes take up most of the outside areas of 
the image. The right and left FFAs are clearly marked 
in red, showing face receptive areas. Next to the FFAs 
are other parts of the inferior temporal lobe, the right 
and left parahippocampal gyri (PHG) that respond to 
inanimate objects, such as houses or shoes. 



FIGURE 14.27 View from the underside of the theme brain, 
showing where the fusiform face area (FFA) is located. 


Haxby and his colleagues (2002) put the results of 
numerous studies together to create a model of face 
perception areas in the brain (Figure 14.29). They pro¬ 
posed a hierarchical system of interconnected brain 
areas to account for both the changeable and invariant 
aspects of face perception that have been discussed in 
this chapter. 

In this model, early visual analysis of facial features 
occurs in the visual cortex, inferior occipital gyrus 
(IOG). The IOG sends information to the superior tem¬ 
poral sulcus (STS) where changeable aspect of faces, 
such as eyes, are processed; from there information 
about eyes is joined with spatial information in the 
intraparietal sulcus (IPS) to generate gaze direction 
information. Information from STS can also be sent to 
the amygdala where social and affective meanings are 
attached and to the auditory cortex where lip move¬ 
ments are registered. Invariant aspects of faces such as 
personal identity are processed in the lateral fusiform 
gyrus (the FFA) that is interconnected with the tempo¬ 
ral lobe where specific information about name and 
biographical data are retrieved. 



FIGURE 14.28 A view of the underside of the brain with 
fusiform face areas (FFA) shown in red and parahippocampal gyri 
(PHG) shown in blue. Source : Caldara et al, 2006. 
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FIGURE 14.29 Model 

of facial perception developed 
by Haxby and colleagues. The 
left side of the model shows 
early visual regions for face 
perception, the center shows 
brain regions for changing 
and non-changing features 
in faces, and the right side 
shows further processing of 
facial features. Source: Haxby 
et ah, 2002. 


4*0 SUMMARY 

Social cognition abilities in human beings are complex 
and multifaceted. They are supported by multiple 
systems of interconnected cortical and subcortical 
areas. During evolution, the simpler valuation and 
behavioral system of the limbic brain was overlaid 
by the growing cerebral cortices that make complex 


cognition possible. It is very likely that our large and 
complex cerebral cortices evolved in part due to selec¬ 
tive pressures brought to bear by the increasing com¬ 
plexity of human society and the demands of social 
cognition. Increasing social and cognitive complexity 
in the environment go hand in hand with increasing 
complexity in the correlated brain systems. 


5.0 CHAPTER REVIEW 
5 ♦ 1 Study questions 

1 Briefly describe what is meant by a theory of 
mind. 

2 According to Frith, what is mentalizing? 

3 Why are shared attention mechanisms important 
for human development? When do they develop? 

4 How do mirror neuron systems differ from other 
neuron systems? What is their role in social 
cognition? 

5 What role does context play in understanding 
intention? 

6 Why is perspective taking a key social skill? 

5.2 Drawing exercises 

1 Label the key brain regions (shown in shades of 
red and pink in Figure 14.30) of the theorized 
frontal mirror neuron system. 



FIGURE 14.30 



































Life is a flame that is always burning itself out , but it catches fire again every time a child is 
born . 


George Bernard Shaw 


Corpus callosum 


Projection fibers 



The mature adult brain contains neural "highways' that are well established and 
course throughout the brain. Until recently, little was known about the develop¬ 
ment of these neural highways in living infants. Using new diffusion tensor 
imaging techniques, the maturation of fiber bundles is studied in 1-4 month old 
infants. The color coding (from blue to red) shows the maturational phase of the 
fiber bundles. Source : Debois et al., 2009. 
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1.0 INTRODUCTION 

In this chapter, we provide an overview of how humans 
grow and develop across multiple stages of life: from 
prenatal to infancy, from child to adolescent. Much of 
our focus will be on early stages of brain and cognitive 
development because the first years of human life rep¬ 
resent a dramatic explosion of neurodevelopmental 
change as babies learn about their world. We will explore 
the roles of nature and nurture in the development of the 


brain and mind, discovering the intricacies of the com¬ 
plex interactions between genetics and experience. 

The field of developmental cognitive neuroscience - 
i.e. the investigation of the maturing brain and its cor¬ 
respondence to human cognition - is a relatively young 
one. The advent of new non-invasive ways to meas¬ 
ure brain function in infants and children has literally 
revolutionized the study of what infants and young 
children understand about the world surrounding 
them. A central focus of the study of the development 
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15. DEVELOPMENT 


of the brain and its relation to behavior relies on the 
combination of multiple techniques and experimental 
approaches in order to elucidate the complexities of the 
mind-brain. 

In this chapter, we will briefly discuss the emerging 
techniques for investigating infant and child develop¬ 
ment. Next, we will trace the anatomical development 
of the brain from prenatal to postnatal stages of life. We 
then focus on brain and cognitive development in the 
first year of life: an explosive time of large-scale changes 
both in brain and in cognition. Next, we track mind- 
brain development through childhood and adolescence. 
We end the chapter looking at the long-term effects of 
early perinatal brain damage with a discussion of brain 
plasticity in childhood. Throughout the chapter, we 
highlight recent empirical investigations of the devel¬ 
opment of the brain and its correspondence to cogni¬ 
tion; however, we add a caveat as this field of study is 



FIGURE 15.1 Techniques for studying brain function in 
infants and children: top panel shows an EEG electrode array; bot¬ 
tom left panel shows an MRI set up for scanning infants and chil¬ 
dren; right panel shows a specialized infant seat for us in MRI 
scanning. Source : Ghislaine Dehaene-Lambertz, with permission. 


relatively new and we are only beginning to understand 
the relationship between the brain and human behavior. 

1.1 New techniques for investigating the 
developing brain 

The emergence of new ways to investigate the human 
brain has been discussed in Chapter 4. Two techniques 
that have been employed in studies of infants and young 
children are electroencephalography/event-related pot¬ 
entials (EEG/ERPs) and functional magnetic resonance 
imaging (fMRI) (Figure 15.1). While these techniques 
have revolutionized the young field of cognitive neu¬ 
roscience, nowhere is the effect felt as strongly as in the 
study of the unfolding of human brain development and 
its correspondence to behavior. Studies of adult behav¬ 
ior and brain function have informed us about how the 
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typically developing brain functions across domains 
such as language, emotion, and memory They also 
inform us about the effects of brain damage or disease. 
However, the pattern of deficit found in adults follow¬ 
ing brain damage differs sharply from the effects when 
brain damage occurs early in life. Therefore, the advent 
of neuroimaging techniques allows us to understand the 
brain regions and cognitive capabilities across cognitive 
domains while it is unfolding in development. 

New and sophisticated methods to investigate ana¬ 
tomical developmental changes throughout life have 
also increased our ability to understand the complex 
patterns of brain development (Figure 15.2). These 
methods allow us to track the development of gray mat¬ 
ter across brain regions as well as to assess connectivity 
patterns across and between the cerebral hemispheres. 

1.2 The mystery of the developing brain: old 
questions and new 

In this chapter, we address some brain questions that 
have been asked for many years. A central question in 


human development is the trading roles of nature and 
nurture. A related issue is to what extent the brain is 
flexible in adapting to new situations in its environ¬ 
ment and to recover from damage. Some new questions 
can be posed that we were previously unable to address 
due to the limitation of our experimental approaches 
or techniques, such as what does a baby know before 
birth? What are the long-standing effects of very early 
brain damage? How do dynamic processes in brain 
development differ across brain regions and hemi¬ 
spheres? We will discuss advances in our knowledge 
about the developmental pathways of three main areas 
of cognition that have been a focus in the field: lan¬ 
guage, executive function, and social cognition. 

2.0 PRENATAL DEVELOPMENT: FROM 
BLASTOCYST TO BABY 

Much of this chapter will be devoted to a discussion of 
brain development and its correspondence to cognition 
during infancy and childhood. Before we begin that 



‘Native' MRI Registered MRI 



Tissue Gray matter 

classification density 



Deformation field Segmentation Cortical thickness Cerebral sulci 


FIGURE 15.2 Image processing pipeline. Top row : a typical image processing pipeline begins with a transfor¬ 
mation of a magnetic resonance (MR) image from the acquisition ('native 7 ) to standardized stereotaxic space; this 
process generates an image that is 'registered 7 with the template brain. The next step involves voxel-wise classifica¬ 
tion of brain tissue into three main classes: gray matter (in red), white matter (in white), and cerebrospinal fluid (in 
green). Each of these binary images (0, tissue absent; 1, tissue present) is then filtered (or smoothed) to generate 
'density 7 images; the image of gray matter (GM) density shown here indicates, at each voxel, the local concentration 
of GM on a continuous scale from 0 to 1 (the 'hotter 7 the color, from blue to red, the higher the value of GM density). 
Bottom row : non-linear registration of the sample image to the template brain allows one to characterize local shape 
differences; the deformation field quantifies such sample-template differences throughout the brain. By combining 
non-linear registration with tissue classification, one can segment automatically various brain structures, such as the 
frontal lobe or the amygdala. Other techniques produce maps of cortical thickness or identify sulci in the subject's 
cerebral cortex. Source: Paus, 2005. 
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discussion, we provide a brief description of the 
processes that occur before birth, during prenatal 
development. While little is known about the sensory, 
perceptual, or cognitive processes of a fetus in utero, 
recent investigations have focused on what a baby 
experiences before birth. These pre-birth experiences 
can be critical for later development. And whether they 
are positive - hearing a mother's voice or her heart¬ 
beat - or negative - experiencing the effects of mater¬ 
nal alcohol abuse - these prenatal experiences can have 
long-standing effects on later cognitive and social devel¬ 
opment. Let's begin our prenatal section with a discus¬ 
sion of gene expression and the role of the environment. 

2.1 Epigenesis 

A central debate in the field of human development is 
the influence of nature versus nurture. Does our genetic 
makeup predetermine who we will become? Or does 
our experience shape who we are? Clearly, both genes 
and the environment have an impact on the develop¬ 
ing human. Does gene expression unfold, followed by 
the development of brain structures and functions that 
later are affected by experience? Or does experience - 
the local environment, whether within a cell, a system, 
or the brain in toto - have an affect on gene expression? 
The interplay between genes and the environment is 
a complex one, with these interactive processes occur¬ 
ring long before birth. Here, we begin the topic of the 
cognitive neuroscience of human development with a 
discussion of the nature of epigenesis. 

Epigenesis, the unfolding of genetic information 
within a specific context, is key to modern ideas about 
development. Different viewpoints on epigenesis 
underlie different perspectives on developmental cog¬ 
nitive neuroscience. Gottlieb and Halpern (2002) have 
drawn a useful distinction between 'predetermined 
epigenesis' and 'probabilistic epigenesis'. Predetermined 
epigenesis assumes that there is a unidirectional causal 
pathway from genes to brain anatomy to changes in 
brain and cognitive function. A hypothetical exam¬ 
ple of this would be if the endogenous expression of 
a gene in the brain generated more of a certain neuro¬ 
chemical. Higher levels of this neurochemical might 
make a particular neural circuit active in the brain, and 
this additional neural activity allows for more com¬ 
plex computations than were previously possible. This 
increased cognitive ability will be evident in behav¬ 
ior as the child being able to pass a task that he or she 
failed at young ages. In contrast, probabilistic epigenesis 
views the interactions between genes, structural brain 
changes, and function as bidirectional (Figure 15.3). 


, Bidirectional influences 

Environment 

(Physical, cultural, social) 

Behavior 

Neural activity 
Genetic activity 

-Individual development-► 

FIGURE 15.3 A systems view of psychobiological develop¬ 
ment. Source : Adapted from Gottlieb and Halpern, 2002. 

Bidirectional interactions mean that not only can genes 
trigger a behavioral change, but also that sensory input 
to the child can change patterns of gene expression. For 
example, we will hear later that newborn infants have 
primitive brain circuits that bias them to look toward 
the faces of other humans (Johnson, 1991). This early 
attention to faces results in some of the neural circuits 
involved in the visual pathways of the baby becoming 
shaped or tuned to process faces. The neuroanatomical 
changes that underlie this shaping process are due to 
differential gene expression. 

2.2 The anatomy of brain development 

Much of early brain development occurs in the first 
weeks following fertilization and we will focus on 
those processes here. Shortly after conception, a fer¬ 
tilized cell undergoes a rapid process of cell divi¬ 
sion, resulting in a cluster of proliferating cells (called 
the blastocyst ) that resembles a tiny bunch of grapes 
(Figure 15.4). After a few days, the blastocyst differ¬ 
entiates into a three-layered structure (the embryonic 
disk). Each of these layers will subsequently differen¬ 
tiate into a major organic system, with the endoderm 
(inner layer) becoming internal organs (digestive, 
respiratory, etc.), the mesoderm (middle layer) becom¬ 
ing skeletal and muscular structures, and the ectoderm 
(outer layer) developing into the skin surface and the 
nervous system (including the perceptual organs). 

The nervous system itself begins with a process 
known as neurulation. A portion of the ectoderm 
begins to fold in on itself to form a hollow cylinder 
called the neural tube (Figure 15.5). 

The neural tube differentiates along three dimen¬ 
sions: length, circumference, and radius. The length 
dimension differentiates into components of the cen¬ 
tral nervous system, with the forebrain and midbrain 
arising at one end and the spinal cord at the other 
(Figure 15.6). The end of the tube that will become 
the spinal cord differentiates into a series of repeated 
units or segments, while the other end of the neural 
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FIGURE 15.5 The neural tube. The overall organization of the 
neural tube emerges soon after closure. The most ventral part of the 
neural tube becomes flattened into a distinct 'floor plate'. The most 
dorsal aspect of the neural tube develops into a tissue known as the 
roof plate. A distinct fissure, the sulcus limitans , forms between the 
dorsal and ventral parts of the neural tube along most of its length. 
Source : Sanes et al, 2006. 



FIGURE 15.4 Blastocyst development. The early processes of 
animal development follow a conserved pattern; after fertilization, a 
series of cleavage divisions divide the egg into a multicellular blas- 
tula. The animal and vegetal poles represent an initial asymmetry 
in the oocyte, and the second axis, dorsal-ventral in this example, 
is established after fertilization. The process of gastrulation brings 
some of the cells from the surface of the embryo to the inside and 
generates the three-layered structure common to most multicellular 
animals. Source : Sanes et al, 2006. 


tube organizes and forms a series of bulges and con¬ 
volutions. Five weeks after conception these bulges 
become protoforms for parts of the brain. One bulge 
gives rise to the cortex, a second becomes the thala¬ 
mus and hypothalamus, a third turns into the mid¬ 
brain, and others form the cerebellum and medulla. 

The distinction between sensory and motor sys¬ 
tems develops along the axis tangential to the surface 
of the neural tube with the dorsal (top-side) becoming 
mainly sensory cortex, and the ventral (bottom-side) 
developing into motor cortex. The various association 
cortices and 'higher' sensory and motor cortices tend 
to arise from the tissue between. 

The radial dimension of the tube differentiates 
into some of the layering patterns in the adult brain. 
Across the radial dimension of the neural tube the 
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FIGURE 15.6 The vertebrate brain and spinal cord develop from the neural tube. Shown here as lateral views 
(upper) and dorsal views (lower) of human embryos at successively older stages of embryonic development (a,b,c). 
The primary three divisions of the brain (a) occur as three brain vesicles or swellings of the neural tube, known 
as the forebrain (prosencephalon), midbrain (mesencephalon), and hindbrain (rhombencephalon). The next stage 
of brain development (b) results in further subdivisions, with the forebrain vesicle becoming subdivided into the 
paired telencephalic vesicles and the diencephalon, and the rhombencephalon becoming subdivided into the meten¬ 
cephalon and the myelencephalon. These basic brain divisions can be related to the overall anatomical organization 
of the mature brain (c). Source : Sanes et al., 2006. 


bulges grow larger and become more distinctive. 
Within these bulges cells proliferate (are born), migrate 
(travel), and differentiate into particular types. The vast 
majority of the cells that will compose the brain are 
born in proliferative zones (Figure 15.7). These zones 
are close to the hollow center portion of the tube (that 
itself later becomes the ventricles of the brain). One of 
these proliferation sites, the ventricular zone , may be 
phylogenetically older (Nowakowski, 1987). Another, 
the subventricular zone , only contributes significantly 
to phylogenetically recent brain structures such as 
the neocortex (i.e. 'new' cortex since it's only found in 
mammals). These two zones yield separate glial (sup¬ 
port and supply cells) and neuron cell lines and give 
rise to different forms of migration. 

Neurons and glial cells are produced by the division 
of cells within the proliferative zone to produce clones 
(a clone is a group of cells which are produced by divi¬ 
sion of a single precursor cell - such a precursor cell is 
said to give rise to a lineage) (Figure 15.8). Neuroblasts 


produce neurons, with each giving birth to a definite 
and limited number of neurons. In some cases neu¬ 
roblasts give rise to particular types of neurons. For 
example, less than a dozen proliferating cells produce 
all the Purkinje cells of the cerebellar cortex, with each 
producing about 10 000 cells (Nowakowski, 1987). 

23 Neural migration 

After young neurons are born, they have to migrate from 
the proliferative zone to the particular region where they 
will be employed in the mature brain. The most com¬ 
mon type of migration involves passive cell displacement. 
In this case young cells are simply pushed farther away 
from the proliferative zone by more recently born cells. 
This gives rise to an outside-to-inside pattern, resulting 
in the oldest cells ending up toward the surface of the 
brain, while the youngest cells are toward the inside. 
This type of migration gives rise to brain structures such 
as the thalamus and many regions of the brainstem. 
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FIGURE 15.7 Fates and migration of neural crest cells. A 
single progenitor cell is injected with a lineage tracer, and its prog¬ 
eny are followed as they migrate out of the neural tube. Some may 
become sensory neurons, while others become Schwann cells or 
neurons of the autonomic nervous system. Environments these cells 
pass through on their migration routes influence their fate choice. 
Source : After Bronner-Fraser and Fraser, 1991; Sanes et al., 2006. 


The second form of migration is more active and 
involves the young cell moving past previously gener¬ 
ated cells to create an ' inside-out ' pattern (Figure 15.9). 
This pattern is found in the cerebral cortex and in 
some subcortical areas that have a laminar structure 
(divided into parallel layers) (Figure 15.10). 

The best studied example of active migration comes 
from the prenatal development of cerebral cortex and 
the radial unit model proposed by Pasko Rakic (1988). As 
mentioned earlier, most cortical neurons in humans are 
generated outside the cortex itself in a region just under¬ 
neath what becomes the cortex, the 'proliferative zone'. 
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FIGURE 15.8 Basic lineage relationships among the cell types 
of the central nervous system of vertebrates. Through a variety of 
cell cultures and in vivo studies, the relationships among the vari¬ 
ous cell classes within the nervous system have been established. 
The early cells of the neural tube have the potential to generate 
an enormous number of progeny and, as a result, are sometimes 
called founder cells or stem cells, which undergo symmetric cell 
divisions to produce additional founder cells as well as progenitor 
cells. (The term stem cells is also used to describe the persistent pro¬ 
genitors found in adult animals.) It is thought that the early founder 
cells also generate progenitor cells that are capable of a more lim¬ 
ited number of cell divisions, and this is the reason that clones of 
progenitor cells labeled late in embryogenesis have fewer progeny. 
Nevertheless, the late progenitor cells are capable of generating 
both neurons and all macroglia, the oligodendrocytes and the astro¬ 
cytes. Although in vitro studies of certain regions of the nervous 
system, particularly the optic nerve, have shown that the lineages 
of astrocytes and oligodendrocytes share a common progenitor, 
known as the 02A glial progenitor, in the spinal cord, motoneurons 
and oligodendrocytes share a common progenitor. Thus, the lineage 
relationships shown may vary depending on the region of the CNS. 
Source : Sanes et al, 2006. 


Recall that the cerebral cortex is much more extensive 
in humans than in most other species. This means that 
these cells must migrate to take up their final locations 
within the cortex. Rakic proposed a 'radial unit model' of 
neocortical differentiation that gives an account of how 
both the regional and the layered structure of the mam¬ 
malian cerebral cortex arise (Rakic, 1988). According to 
his model, the laminar organization of the cerebral cor¬ 
tex is determined by the fact that each relevant prolif¬ 
erative unit gives rise to about one hundred neurons. 
The progeny from each of these proliferative units all 
migrate up the same radial glial fiber, with the latest 
to be born travelling past their older relatives. A radial 
glial fiber is a long process that stretches from top to 
bottom of the cortex and originates from a glial cell. 
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FIGURE 15.9 Histogenesis in the cerebral cortex proceeds through three stages. In the first stage of histogenesis, the wall of the cerebral 
cortex is made up of the progenitor cells, which occupy the ventricular zone (VZ). In the next stage of development, the first neurons exit the 
cell cycle (red) and accumulate in the preplate, adjacent to the pial surface. The neurons of the preplate can be divided into the more superficial 
Cajal-Retzius cells and the subplate cells. In the next stage of cortical histogenesis, newly generated neurons (red) migrate along radial glial 
fibers to form a layer between the Cajal-Retzius cells and the subplate. This layer is called the cortical plate, and the majority of the neurons in 
the cerebral cortex accumulate in this layer. Source : Sanes et ah, 2006. 


Radial glial fibers effectively act like a climbing rope to 
ensure that neurons produced by one proliferative unit 
all contribute to one radial column within the cortex 
(Figure 15.11). 

There are some interesting consequences of the 
radial unit model for species differences in the cerebral 
cortex. Rakic (1988) points out that greater cell division 
at the proliferative unit formation stage would increase 
the number of cortical columns and hence the total area 
of cortex that results. In contrast, an additional single 
round of division at a later stage, from the prolifera¬ 
tive zone, would only increase the size of a column by 
one cell (about 1 percent). This corresponds well with 
the fact that there is very little variation between mam¬ 
malian species in the general layered structure of the 


cortex, while the total surface area of the cortex can 
vary by a factor of 100 or more between different spe¬ 
cies of mammal. It seems likely, therefore, that species 
differences originate (at least in part) in the number of 
'rounds' of cell division that are allowed to take place 
within and across regions of the proliferative zone. 

Thus, in the early weeks of gestation, the embryo 
undergoes complex processes that form the basis for 
the central nervous system. It is important to note that 
prenatal brain development is not a passive process 
involving the unfolding of genetic instructions. Rather, 
from an early stage interactions between cells are criti¬ 
cal, including the transmission of electrical signals 
between neurons. In one example, patterns of spon¬ 
taneous firing of cells in the eyes (before they have 
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FIGURE 15.10 Histogenesis of pyramidal neurons of the deep layers, V and VI. (a) After the birth and migration of the 
Cajal-Retzius cells and the subplate cells, the next neurons to be generated in the cortex are the pyramidal neurons of the deep 
layers, V and VI, whose axons project to subcortical targets, (b) The next neurons to be born are the local interneurons in layer IV 
of the cortex, (c) Finally, the pyramidal cells of the upper layers, II and III, are generated. They send axons to other cortical areas. 
Source : Sanes et al, 2006. 


opened in development) transmit signals that appear 
to specify the layered structure of the visual thalamus, 
the lateral geniculate nucleus (LGN) (see Shatz, 2002; 
O'Leary and Nakagawa, 2002). Thus, waves of firing 


intrinsic to the developing organism may play an 
important role in specifying aspects of brain structure 
long before sensory inputs from the external world 
have any effect. 
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FIGURE 15.11 The radial unit model of Rakic (1988). Radial 
glial fibers span from the ventricular zone (VZ) to the cortical 
plate (CP) via a number of regions: the intermediate zone (IZ) and 
the subplate zone (SP). RG indicates a radial glial fiber, and MN a 
migrating neuron. Each MN traverses the IZ and SP zones that con¬ 
tain waiting terminals from the thalamic radiation (TR) and cortico- 
cortico afferents (CC). As described in the text, after entering the 
cortical plate, the neurons migrate past their predecessors to the 
marginal zone (MZ). Source : Rakic et al., 2009. 


2.4 Nature and nurture revisited 

As we mentioned at the beginning of this section, the 
role of the prenatal environment in the unfolding of 
genetic instructions and brain development can have 
long-lasting effects, both good and bad. The developing 
infant is susceptible to events occurring within this envi¬ 
ronment. One such event is the incursion of a teratogen. 
A teratogen is defined as any environmental agent that 
causes damage during the prenatal period. Examples of 
teratogens are prescription and even non-prescription 
drugs, caffeine found in coffee and soft drinks, illegal 
drugs such as cocaine and heroin, tobacco and marijuana 
products, and alcohol. The effects of the teratogen(s) can 
be complex depending on the dosage level, the time it 
occurs during prenatal development, and the genetic 
makeup of the mother, since some individuals are more 


susceptible than others. Prenatal counseling for mothers- 
to-be providing education regarding potential sources of 
teratogens has helped to reduce the occurrences of brain 
damage due to teratogens. 

The prenatal brain is particularly susceptible to the 
effects of alcohol. Alcohol abuse by the mother during 
pregnancy results in long-term deficits in cognition, 
language, and social development called fetal alcohol 
syndrome (FAS) (Jones and Smith, 1973; Jones, 1975). A 
recent brain mapping study of children, teenagers, and 
young adults with severe FAS showed gray matter den¬ 
sity differences in FAS individuals as compared to age- 
and gender-matched controls. Specific findings were 
reduced gray matter density in frontal and parietal areas 
and increased density in temporal and inferior parietal 
lobe regions (Figure 15.12). As we will see in later sec¬ 
tions of this chapter, these brain areas mature through¬ 
out childhood and into late adolescence; therefore these 
gray matter density differences in FAS individuals indi¬ 
cate that prenatal exposure to alcohol has a resounding 
and long-lasting impact on brain development and cog¬ 
nitive development throughout the lifespan. 

Longitudinal studies assessing the long-term effects 
of smoking cigarettes or marijuana during pregnancy 
have provided new evidence about their impact on a 
child's cognitive development. Fried and colleagues 
(Fried et al., 2003) have followed a cohort of children in 
Canada from birth through young adulthood. Using 
neuropsychological test batteries to assess cognitive 
functions like verbal intelligence, visuo spatial process¬ 
ing, language abilities, attentional function. Fried and 
colleagues found that there are early-occurring (by age 3) 
and long-lasting cognitive impairments caused by the 
mother smoking either tobacco or marijuana during 
pregnancy. The specific effects of prenatal exposure to 
cigarette smoke differ sharply from exposure to mari¬ 
juana, although both cause harm. Exposure to cigarette 
smoke resulted in lower general intelligence in the chil¬ 
dren coupled with deficits in auditory function and 
verbal working memory that continued from early 
childhood (age 3) through adolescence (age 16) (Fried 
et al., 2003). Exposure to marijuana smoke resulted in 
no general intelligence deficit; however, executive func¬ 
tions, such as attention and working memory, were 
impaired in these children and, in particular, visual 
processes such as visual integration, analysis, and rea¬ 
soning. Again, these impairments were observed early 
(age 3) and continued through the teenage years. 

The cognitive impairments reflected by the neu¬ 
ropsychological test batteries implicated specific brain 
regions for further study of the effects of prenatal expo¬ 
sure to cigarette and marijuana smoke. Visuospatial 
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FIGURE 15.12 Differences in gray matter density between children with fetal alcohol syndrome (FAS) and typically developing con¬ 
trols. Warm (red, yellow) colors represent positive differences, indicating an increase in gray matter density (and thus a decrease in normal 
'pruning') in those regions as compared to controls. Note that the children with FAS have much increased densities in temporal lobe regions, 
particularly in the left hemisphere. (Adapted, with permission, from Sowell et al., 2002.) Source : Toga et al., 2006. 




FIGURE 15.13 Effects of prenatal use of marijuana smoke on young adults (18-22 years) measured using 
fMRI. Frontal lobe circuits engaged in a task tapping visuospatial working memory systems, with reduced right 
hemisphere activity (upper panel) and increased left hemisphere activity (lower panel) as compared to an age- 
matched control group. Source: Smith et al., 2006. 


integrative processes tap frontal lobe regions in adults 
(see Chapter 6). Fried and colleagues (Smith et ah, 2006) 
continued their investigation into the long-term effects 
of prenatal exposure to marijuana using a neuroimaging 
technique, fMRI, with a sample of young adults (18-22 
years) from the Canadian cohort with prenatal expo¬ 
sure to marijuana smoke. Results of the study revealed 
a differing pattern of neural activity in frontal lobe 


regions that are engaged in a visuospatial task. Specific 
findings were reduced activity in right hemisphere 
regions (Figure 15.13, upper panel) and increased 
activity in left hemisphere regions (Figure 15.13, 
lower panel). The authors interpreted these findings 
as indicating that right hemisphere neural circuitry 
engaged in tasks that tap visuospatial short-term mem¬ 
ory are less active in children with prenatal marijuana 
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smoke exposure. While it is difficult to know at this 
stage of the research just why left hemisphere activity 
was increased, it could be that left hemisphere regions 
were recruited as a compensatory mechanism due to 
the decreased neural activation in the right hemisphere. 
More research is clearly needed in order fully to under¬ 
stand the effects of prenatal cigarette and marijuana 
smoke exposure. However, the work of Fried and col¬ 
leagues provide compelling evidence that the damage 
appears early in a child's cognitive development and is 
very long lasting. The clear message from these stud¬ 
ies is that early insult caused by a teratogen, such as 
cigarette or marijuana smoke, can produce lifelong cog¬ 
nitive impairment and may correspond to differing pat¬ 
terns in cortical development. 


2*5 Prenatal hearing experience: voice and 
music perception before birth 

What do babies know before they are born? Is it impor¬ 
tant for a mother-to-be to talk to her unborn baby? 
Read to her baby? Sing to her baby? Is there an impact 
on later language, music, and cognitive function? In 
other words, what are the perceptual abilities of an 
unborn child and how do they relate to later cognitive 
development? This question has intrigued develop¬ 
mental psychologists for at least a hundred years (see 
Kisilevsky and Low, 1998, for a review), but system¬ 
atic investigations of fetal perception did not get under 
way until the 1980s. How do you measure a fetal 
response to sounds? Usually, the investigators measure 
heart rate changes and sometimes body movements 
in response to differing types of sounds. These early 
studies provided evidence that by approximately 
30 weeks' gestational age, a fetus hears and responds 
to simple sounds such as bursts of white noise. By 
37-42 weeks, a fetus can discriminate between types of 
speech sounds (such as vowels, consonant-vowel syl¬ 
lables) (Lecanuet et ah, 1987; Groome et ah, 1999). 

The finding that a fetus can both hear and discrimi¬ 
nate between sounds before birth has led to investi¬ 
gations of what a fetus knows about specific sounds, 
namely, his or her own mother's voice. DeCasper and 
colleagues studied the listening preferences of newborn 
infants in a series of investigations in the 1980s. They 
found that newborns prefer their mother's voice to that 
of a female stranger (DeCasper and Fifer, 1980). These 
results led to a debate regarding the bases of speech 
perception: does a newborn's preference for his or her 
own mother's voice reflect a genetically pre-wired dis¬ 
position for human language? Or does the newborn's 


preference reflect prenatal experience? A recent work 
by Kisilevsky and colleagues (Kisilevsky et ah, 2003) 
advanced the results provided by DeCasper with more 
specific evidence that a late-term fetus recognizes the 
voice of his or her mother over the voice of a female 
stranger. They reported that the fetal heart rate increases 
when listening to the mother's voice and decreases 
when listening to a stranger's voice. While the findings 
of Kisilevsky and colleagues (Kisilevsky et ah, 2003) do 
not resolve the nature versus nurture debate regarding 
the genetic or experience basis of a newborn's speech 
perceptual acuity, they provide intriguing evidence 
that a fetus can not only discern between familiar and 
unfamiliar voices, but suggests that before birth, there 
is a specific preference for the mother's voice. 

There has recently been an increased focus on the 
effects of prenatal exposure to music. This focus is largely 
due to claims that prenatal exposure to music produces 
increases in intelligence or cognitive function as the 
baby develops. These claims are based on anecdotal 
information, however, with no rigorous scientific basis. 
Let's examine the evidence of what we do know about 
music perception at or near birth: there have been many 
infant studies investigating music perception abilities 
in the first months of life. Trehub and colleagues report 
that infants as young as 2 months old perceive rhythmic 
patterns in music. Further, Trehub reports that there are 
many parallels between early infant musical knowledge 
and adult patterns that appear to support some notion 
of an innate or at least a biological bias for music percep¬ 
tion (see Trehub and Hannon, 2005, for a review). 

Little is known about musical perception before 
birth. However, Kisilevsky and colleagues (Kisilevsky 
et ah, 2004) have reported that late-term (33-37 weeks' 
gestational age) fetuses can discriminate between 
changes in tempo in musical sequences. Importantly, 
earlier term fetuses (28 weeks) did not show this effect, 
leading Kisilevsky to conclude that there are develop¬ 
mental changes in utero in music perception, with later 
term fetuses discriminating features in music that ear¬ 
lier term fetuses did not discriminate. Cumulatively, 
the research carried out by Kisilevsky and colleagues 
provides evidence that some higher order auditory 
perception for maternal voice and for music is not 
only occurring before birth, but is changing, maturing 
in its nature. 

Does evidence that a fetus perceives human voice 
and music before birth rule out the idea of a genetic 
predisposition for language and music? Not really. 
The elucidation of the complex interactions between 
genetic expression and experience that underlie human 
speech and music perception are still being elucidated 
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in the field of developmental cognitive neuroscience. 
New techniques and experimental designs are being 
developed in order to address this very important 
question about human development. 


3*0 THE DEVELOPING BRAIN: 

A LIFETIME OF CHANGE 

3 .1 The rise and fall of postnatal brain 
development 

While the overall appearance of the newborn human 
brain is rather similar to that in adults, and most neu¬ 
rons have already reached their final locations, a number 
of substantive additive changes occur during postnatal 
development of the brain. Specifically, brain volume 
quadruples between birth and adulthood, an increase 
that comes from a number of sources, but generally not 
from additional neurons. The generation and migration 
of neurons takes place almost entirely within the period 
of prenatal development in the human. Although there 
may be some addition of neurons in the hippocampus 
and elsewhere, the vast majority are present by around 
the seventh month of gestation (Rakic, 1995). 

Perhaps the most obvious change during post¬ 
natal neural development is the increase in size and 
complexity of the dendritic trees of most neurons. An 
example of the dramatic increase in dendritic tree 
extent during human postnatal development is shown 
in Figure 15.14. While the extent and reach of a cell's 
dendritic arbor may increase dramatically, it also often 
becomes more specific and specialized. 

In addition to the more extensive processes involved 
in the inputs and outputs of cells, there is a steady 
increase in the density of synapses in most regions of 
the cerebral cortex in humans (Huttenlocher et al., 1982; 
Huttenlocher, 1990,1994). The process of creating new 
synapses, synaptogenesis , begins approximately around 


the time of birth for all cortical areas studied to date, 
with the most increases, and the final peak density, 
occurring at different ages in different areas. For exam¬ 
ple, in the visual cortex there is rapid synaptogenesis 
at 3 to 4 months, and the maximum density of around 
150 percent of that seen in adult humans is reached 
between 4 and 12 months. In contrast, while synap¬ 
togenesis starts at the same time in a region of the pre¬ 
frontal cortex, density increases much more slowly and 
does not reach its peak until well after the first year. 

Another additive process is myelination. Myelination 
refers to an increase in the fatty sheath that surrounds 
neuronal processes and fibers, that increases the effi¬ 
ciency of electrical transmission. Because myelination 
continues in cortical areas for many years after birth, 
there has been speculation about its role in behavioral 
development (Yakovlev and Lecours, 1967; Parmelee 
and Sigman, 1983; Volpe, 1987). Owing to the increased 
lipid content of the brain caused by myelination of fib¬ 
ers, structural MRI images can reveal a clear gray-white 
matter contrast and this allows quantitative volume 
measurements to be made during development (see 
Sampaio and Truwit, 2001). While some controversy 
remains about the interpretation of images from infants 
under 6 months, there is consensus that the appearance 
of brain structures is similar to that of adults by 2 years 
of age, and that all major fiber tracts can be observed 
by 3 years of age (Huttenlocher and Dabholkar, 1997; 
Bourgeois, 2001). Changes in the extent of white mat¬ 
ter are of interest since they presumably reflect inter¬ 
regional communication in the developing brain. While 
increases in white matter extend through adolescence 
into adulthood, particularly in frontal brain regions 
(Huttenlocher et al., 1982), the most rapid changes 
occur during the first 2 years. Myelination appears to 
begin at birth in the pons and cerebellar peduncles and, 
by 3 months, has extended to the optic radiation and 
splenium of the corpus callosum. Around 8-12 months 
the white matter associated with the frontal, parietal, 
and occipital lobes becomes apparent. 
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FIGURE 15.14 Dendritic arboriza¬ 
tion. A drawing of the cellular structure of 
the human visual cortex based on Golgi 
stain preparations from Conel (1939-1967). 
Source: Quartz, 1999, from Conel, 1953. 
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Surprisingly, human postnatal brain development 
also involves some significant regressive events. One 
quantitative neuroanatomical measure of a regressive 
event is the density of synapses, where there is a period 
of synaptic loss or pruning (Huttenlocher, 1990, 1994). 
Like the timing of bursts of synaptogenesis, and the 
subsequent peaks of density, the timing of the reduc¬ 
tion in synaptic density appears to vary between corti¬ 
cal regions, at least in humans. For example, synaptic 
density in the visual cortex begins to returns to adult 
levels after about 2 years, while the same point is not 
reached until adolescence for regions of the prefrontal 
cortex. Huttenlocher (1990,1994) suggests that this ini¬ 
tial overproduction of synapses may have an important 
role in the apparent plasticity of the young brain. 

Thus, the rise and fall developmental sequence is 
seen in a number of different measures of neuroana¬ 
tomical and physiological development in the human 
cortex. However, we need to bear in mind that not 
all measures show this pattern (e.g. myelinization) 
and that measures of synaptic density are static snap¬ 
shots of a dynamic process in which both additive and 
regressive processes are continually in progress. 

While most of this section has focused on the cerebral 
neocortex, other brain structures, such as the hippocam¬ 
pus and cerebellum, also show some postnatal devel¬ 
opment. Indeed, the postnatal development of some 
subcortical structures (such as the hippocampus, cere¬ 
bellum, and thalamus) poses something of a paradox; 
on the one hand there is much behavioral and neural 
evidence to indicate that these structures are function¬ 
ing at birth, while on the other they all show some 
evidence of postnatal development and/or functional 
re-organization. One resolution of this puzzle is that as 
the cerebral cortex develops postnatally, its interactions 
with subcortical regions undergo certain changes. Thus, 
while some subcortical structures are capable of func¬ 
tioning relatively independently of the cortex early in 
life, the increasing development of the cortex requires 
some structural and functional adjustment. 

3.2 Regional differences in brain development 

As compared to other species, humans take a very 
long time to develop into independent creatures. 
Human postnatal cortical development, for example, 
is extended roughly four times as long as non-human 
primates. The 'down' side of this slow development is 
that there are many years during which a child is highly 
dependent on the care provided by family members. 
The 'up' side of this protracted developmental timeta¬ 
ble is that the human brain has far more opportunity for 


experience, and interactions with others, to shape and 
mold its development. 

As suggested above, the rise and fall pattern of addi¬ 
tive and regressive events occurs at different time frames 
in regions and lobes in the human brain. These events 
are heavily experience driven and reflect synapse for¬ 
mation and dendritic arborization that are due to cogni¬ 
tive and sensory development, learning, and integrative 
processes that occur throughout infancy and childhood 
and continue through the teen years. A time course 
of brain development from conception to late teens is 
presented in Figure 15.15 (Casey et ah, 2005). Prenatal 
changes are shown on the left side of the figure and 
largely reflect neurulation, cell proliferation, and migra¬ 
tion processes. Postnatal changes reflect developmental 
processes such as synaptogenesis and dendritic arbori¬ 
zation. Sensory areas for processing visual and auditory 
information, for example, develop earlier than frontal 
lobe regions such as the prefrontal cortex for processing 
executive functions (Casey et al., 2005). 

How can we objectively measure brain develop¬ 
ment over the lifespan? One way in which to track 
developmental changes in the brain is to measure 
gray matter density across regions in the cortex. A 
large cross-sectional study of 176 subjects ranging in 
age from 7 to 87 years provided evidence for differ¬ 
ent patterns of gray matter density decrease across the 
lifespan (Toga et al., 2006). In the superior frontal sul¬ 
cus, gray matter density decreases rapidly beginning 
in adolescence. However, in the superior temporal 
sulcus, gray matter density decreases more gradually 
through life (Figure 15.16). 

While we are beginning to map out the develop¬ 
mental patterns for brain regions across the lifespan, a 
key issue that is being addressed is the notion of indi¬ 
vidual differences in how the brain matures and the 
correspondence to cognitive development. In other 
words, what is the relationship between brain and 
behavior? Do differing patterns of brain development 
reflect different levels of intellectual ability? These 
questions were addressed in a recent neuroimaging 
investigation of gray matter density in a large sam¬ 
ple of more than 300 children and adolescents (Shaw 
et al., 2006). The sample was a priori divided into three 
groups based on their performance on an IQ battery 
of tests: 'Superior' with IQ ranging from 121 to 149; 
'High' with IQ ranging from 109 to 120; and 'Average' 
with IQ ranging from 83 to 108. The results indicated 
that there are, indeed, differing patterns of brain 
change corresponding to overall level of intelligence 
(Figure 15.17). The notable finding was that high IQ 
was associated with thinner cortex, especially in frontal 
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Developmental course of human brain development 


Experience-dependent synapse formation 



FIGURE 15.15 Developmental course of human brain development. The human brain undergoes dramatic 
changes in both its structural architecture and functional organization that reflect a dynamic interplay of simultane¬ 
ously occurring progressive and regressive events. Although the total brain size is about 90 percent of adult size 
by age 6 years, the brain continues to undergo dynamic changes throughout adolescence and well into adulthood. 
Figure 15.15 illustrates some of these developmental changes, including proliferation and migration of cells mostly 
during fetal development, regional changes in synaptic density during postnatal development, and protracted devel¬ 
opment well into adulthood. Current non-invasive neuroimaging methods do not have the resolution to delineate 
which of these processes underlies observed developmental changes beyond gray and white matter subcomponents. 
(Adapted from Thompson and Nelson, 2001.) Source : Casey et ah, 2005. 


and temporal lobe areas, in early childhood. By late 
childhood, the opposite pattern was found, with high 
IQ associated with thicker cortex. 

The important finding of this study was that there 
was a differing pattern of cortical development in fron¬ 
tal lobe regions for children in the 'Superior' group as 
compared to either of the other groups. Specifically, 
there were differences in the dynamic rate of cortical 
thickening and thinning throughout early childhood 
and into adolescence and early adulthood. The authors 
concluded that differences in gray matter density in 
and of itself did not lead to children with superior 
intellectual abilities; rather they suggested that the 
dynamic properties of development of the cortex cor¬ 
responded to level of intelligence, perhaps enabling 
the child to extract more information from his environ¬ 
ment. Open issues raised by this study are dynamic 
changes in brain growth patterns due to genetic pre¬ 
dispositions in the 'Superior' group? Or do they reflect 
differences in the environment? Or do they reflect a 
combination of both genetic influences and experience? 

These findings from recent studies of brain devel¬ 
opment throughout the lifespan are helping us pre¬ 
pare new models for human cognitive growth and the 
correspondence to cognition. These studies are still 


very early work in an ongoing series of investigations 
of complex brain developmental processes. Although 
this investigation is still in its early stages, results to 
date provide evidence that the brain is changing in 
dynamic ways throughout early childhood and into 
adulthood. 


4.0 DEVELOPING MIND AND BRAIN 

We have seen that the brain undergoes significant 
developmental changes throughout childhood. An 
important question to address in the field of devel¬ 
opmental cognitive neuroscience is how these brain 
changes reflect development of cognitive processes. 
New techniques are allowing us to begin to map brain 
development and the correspondence to cognition. 
The emergence of longitudinal multidisciplinary stud¬ 
ies combining neuroimaging studies of cortical change 
and behavioral studies of performance on cognitive 
tasks will provide us with better ways to asses the rela¬ 
tionship between brain development and cognition. In 
this section, we will discuss what we know at present 
about the relation between brain development and 
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Superior frontal sulcus 
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FIGURE 15.16 Mapping brain change over time. Brain changes in development can be identified by fitting time-dependent 
statistical models to data collected from subjects cross-sectionally (i.e. across a group of subjects at a particular time), longitudinally 
(i.e. following individual subjects as they aged), or both. Measurements such as cortical thickness are then plotted onto the cortex 
using a color code, (a, b) Trajectory of gray matter loss over the human lifespan, based on a cohort of 176 subjects aged 7 to 87 years 
(Sowell et al., 2003). Plots superimposed on the brain in (b) show how gray matter density decreases for particular regions; (a) high¬ 
lights example regions in which the gray matter density decreases rapidly during adolescence (the superior frontal sulcus) or fol¬ 
lows a more steadily declining time course during the lifespan (the superior temporal sulcus) (c, d). Source: Toga et al., 2006. 
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FIGURE 15.17 Correlations between IQ and cortical thickness, (a) Pearson's correlations for all 307 subjects were generally positive 
and modest (p > 0.05), with r between 0 and .10 (green/yellow), except in the anterior temporal cortex (which showed a negative correla¬ 
tion, with r between 0 and —1; blue/purple), (b) Correlations in different age groups showed that negative correlations were present in 
the youngest group, indicating that higher IQ was associated with a thinner cortex particularly in frontal and temporal regions. The rela¬ 
tionship reverses in late childhood with most of the cerebral cortex correlating positively with IQ. Source : Shaw et al., 2006. 


FRONTIERS OF COGNITIVE NEUROSCIENCE 


How babies learn 



FIGURE 15.18 Jenny Saffran, PhD, Waisman Center, 
University of Wisconsin, Madison, WI, USA. 

Speech is by far the most complex sound that humans 
decode. A central question for scientists who study 
human development is: How do infants learn to decode 
this complex signal? 

You may have learned a new language in school dur¬ 
ing your adolescent or teen years. You were guided in 
this process by several factors that are not present for 
infants learning their first language: first, you were aware - 
conscious - that you were learning a new language; sec¬ 
ond, you could relate the new information to your existing 


knowledge of your first language (for example, you could 
learn that agua was the Spanish word for water); and third, 
you had a lot of existing knowledge about the structure of 
your language (for example, you know that individual 
sounds (phonemes) combine to form syllables and words). 

Infants have none of this metalinguistic knowledge. 
They are simply highly motivated to interact with their 
family members and caregivers. So one wonders, how 
do infants learn to decode speech? How do they learn 
that the nearly continuous speech stream contains dis¬ 
crete words and phrases? In other words, how do infants 
extract word boundary information in speech? 

Dr. Jenny Saffran at the University of Wisconsin, 
Madison, has some fascinating explanations for how 
infants accomplish this. Dr. Saffran theorizes that one 
important aspect of infant language learning is that infants 
unconsciously extract statistical regularities in the speech 
stream (Saffran et al, 1996; Aslin et al, 1998). Here is how 
she proposes it happens: within any given language, there 
are sound patterns that occur with some level of prob¬ 
ability. Although all languages have many individual 
phonemes, sounds that are specific to that language, they 
form a finite set. And the way they are combined to form 
words—their phonology—is also constrained. Dr. Saffran 
theorizes that infants extract the transitional probability (TP) 
of sound patterns in speech (Figure 15.19). That is, the 
probability that a certain sound pattern will follow any 
given sound pattern, conditioned on the probability of the 
first sound pattern. 
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T p _ p(Yix) = fre q uenc y ( xy ) 

frequency (X) 

FIGURE 15.19 Schematic of an artificial grammar 
used by Saffran. Source : adapted from Saffran et al, 1996, with 
permission. 

For example, in English, the syllable 'ba' frequently 
is followed by the syllable 'by' to form the word 'baby' 
and this combination occurs with a much higher prob¬ 
ability than the combination of 'ba' and 'sin' to form the 
word 'basin'. The TP of these combined syllables, then, 
is the probability of 'by' given the probability of 'ba', and 
the probability of 'sin' given the probability of 'ba'. And 
though the probability of the occurrence of 'ba' in the 
speech stream is the same, the TP differs sharply when 
combined with 'by' versus 'sin'. 

According to Dr. Saffran, infants extract these prob¬ 
abilities and learn to form word boundaries using statis¬ 
tical learning mechanisms. Extracting the TPs in speech 
helps the infant to detect word boundaries and eventu¬ 
ally to acquire language. 

The work of Dr. Saffran and her colleagues has lit¬ 
erally revolutionized the way developmental and lan¬ 
guage researchers think about infant learning processes. 
It has sparked a wave of new investigations, includ¬ 
ing new studies from the Infant Learning Laboratory 
directed by Dr. Saffran. Here are some 'threads' that 
are running through her current research in her Infant 
Learning Laboratory: 

1. Natural speech: In her earlier work. Dr. Saffran used 
simplified stimulus sets and grammars to elucidate 
how infants develop speech perception. In her cur¬ 
rent set of studies, she is investigating infant learning 
processes using natural language stimuli. Results so far 
show that infants are as good or better at statistical 
learning in naturalistic speech (Pelucchi et al., 2009). 

2. Real time learning: The Saffran lab is looking at how 
infants learn in 'real time' by investigating the effects 
of statistical properties as they are occurring. Language 


comprehension in adults is facilitated by statistical 
information - knowing what is likely to come next. 
Dr. Saffran is currently testing the hypothesis that 
infants, too, make use of probabilistic information 
during language comprehension, as measured by 
eye-gaze tracking. 

3. Integrating across clinical populations: The Saffran 
lab has been investigating how typically developing 
infants and children learn language. Now they are 
expanding their research to see if infants and children 
with language impairment or hearing loss learn lan¬ 
guage in a similar way (Evans et al., 2009). 

Although Dr. Saffran does not study the brain bases 
for these early-established infant learning mechanisms, 
her work raises intriguing questions: where are these 
mechanisms located in the infant brain? Do they rep¬ 
resent domain general learning mechanisms that are 
invoked for language acquisition, but are also used in 
other cognitive development? Or are there multiple - and 
perhaps parallel - systems of learning that unite to form 
the scaffolding for knowledge and conceptual represen¬ 
tation in infants and children? These questions remain 
open in the field of the cognitive neuroscience of child 
development so we shall have to wait to see what new 
studies tell us about brain mechanisms underlying infant 
learning! 
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child developmental processes. While we highlight 
some recent studies, it is important to note that the 
field of developmental cognitive neuroscience is a very 
young one. The first fMRI studies of children were 
published in 1995 by Casey and colleagues (Casey 
et al., 1995). Thus, while the findings we present here 
are informative, the answer to our questions regard¬ 
ing the neural bases of language acquisition, cognitive 
control processes, and social cognition are still being 
discovered in laboratories throughout the world. 


We have discussed a functional framework for under¬ 
standing the processes of cognition. Developmental 
cognitive neuroscientists are seeking to understand 
how these systems and processes that are observed in 
healthy adults (Figure 15.20) are developed and formed 
in infancy, childhood, and through adolescence. Once 
again, the question of the role of nature versus nurture 
is debated in the field. Is the human brain pre-wired for 
language? Face perception? Or are these processes based 
on experience throughout life? Since these topics are of 
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central interest in the study of human development, we 
will focus on three general areas of developmental cog¬ 
nitive neuroscience investigations that may shed light on 
these issues: the emergence of language; the development 
of cognitive control mechanisms; and the development of 
social cognition, with a specific focus on face perception. 
In the following section, we provide a brief summary of 
research to date on these topics in infants during the first 
year of life. Next, we present findings on these topics for 
older children and adolescents. Last, we review the effects 
of early (perinatal) brain damage on these systems. 

4*1 The first year of life: an explosion of 
growth and development 

The human brain increases fourfold in size from new¬ 
born to adult. Many of the dynamic changes that 
occur in development happen during the first year of 
life. During this 12-month span, an infant develops 
from a tiny creature with few voluntary movements to 
a busy toddler smiling, reaching for attractive objects, 
producing many speech sounds, crawling, and even 
walking as he or she explores the world. 


4 A A Developing the linguistic brain: infant 
language capabilities 

Remember the studies showing that babies can hear 
and discriminate their mother's voice before birth? 
Studies like these have provided compelling evidence 
that a newborn infant already has experience with 
human language. We have discussed the nature versus 
nurture debate regarding the genetic predisposition 
for language versus the role of experience. This debate 
has been influential in the field of developmental cog¬ 
nitive neuroscience, with many studies investigating 
just what an infant knows about language. Most stud¬ 
ies of young infants (less than 12 months old) focus 
on the classes and categories of speech sounds: pho¬ 
nology. Studies with older infants and children also 
investigate semantic (meaning-based) and syntactic 
(grammar-based) knowledge. 

Is language 'biologically special'? One way to begin 
to address this question is to see whether young babies 
are specifically sensitive to human speech. If there are 
specific neural correlates of speech processing observa¬ 
ble very early in life, this may indicate language-related 
neural processing prior to significant experience. 
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FIGURE 15.20 A functional framework for development, adapted from the general functional framework 
presented in Chapter 2. 
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One example of this approach concerns the ability 
to discriminate speech-relevant sounds such as pho¬ 
nemes (see Chapter 7). Each human language has a set 
of sounds that map onto individual phonemes, which 
typically are conscripted within slashes: /p/, for 
example, to reflect the sounds (phones) that map onto 
the phoneme /p/. Recall the 'lack of invariance' prob¬ 
lem discussed in Chapter 7: the differing phonemes 
that are articulated before and after the articulation of 
/p/ affect its acoustic features. Thus, there is not a sin¬ 
gle invariant physical property that uniquely defines 
/p/. Rather, the representation of the phoneme /p/ 
must rely on some abstract (not just physical) features. 
This aspect of human speech has been exploited in 
speech perception studies where phonemes that differ 
in a single feature are prepared using speech synthe¬ 
sizing software to create a series of sounds that differ 
in graded steps between, for example, the phonemes 
/b/ and /p/, which differ only in their initial voicing 
(vocal chord vibration). 

As English-speaking adults, if we were to listen to 
a graded phonetic transition from speech sounds 'ba' 
to 'pa', we would perceive the intermediates between 
/ba/ and /pa/ as being either one or the other. In other 
words, we show a categorical boundary between the 
two. Behavioral experiments have revealed that young 
babies also show enhanced (categorical) discrimination 
at phonetic boundaries used in speech such as /ba/ 
/pa/. That is, a graded phonetic transition from /ba/ 
to /pa/ is also perceived as a sudden categorical shift 
by infants. These observations initially caused excite¬ 
ment as evidence for a human speech perception- 
specific detection mechanism in humans. However, 
more recent research has shown that other species, 
such as chinchillas, show similar acoustical discrimi¬ 
nation abilities. This indicates that this ability may 
merely reflect general characteristics of the mam¬ 
malian auditory processing system and not an initial 
spoken language-specific mechanism (see Werker and 
Vouloumanos, 2001). 

Intriguingly, and unlike adults, human infants can 
initially discriminate a very wide range of phonetic con¬ 
structs, including those not found in their native lan¬ 
guage. For example, Japanese infants, but not Japanese 
adults, can discriminate between 'r' and T sounds. 
However, this ability becomes restricted to the phonetic 
constructs found in their native language by around 10 
months of age. These findings might reflect early speech 
perceptual processes that take into account the physi¬ 
cal or acoustic features in all speech sounds in early 
infancy, developing later into mechanisms with less reli¬ 
ance on the physical aspects and more on the abstract 


representations of phonemes in their native language. 
In this way, the role of experience has a strong hand in 
shaping an infant's language knowledge. 

If brain correlates of this process could be iden¬ 
tified, it may be possible to study the mechanisms 
underlying this speech-specific selective decrease of 
sensitivity. Event-related potentials (ERPs) have been 
used to investigate this question. When components 
of ERP differ in both latency (following the event) 
and spatial resolution, we may be confident that dif¬ 
ferent neural circuitry is being activated in the brain. 
Dehaene-Lambertz and Dehaene (1994) presented 
babies with trials in which a series of four identical 
syllables (the standard) was followed by a fifth that 
was either identical or phonetically different (devi¬ 
ant). They time-locked the ERP to the onset of the 
syllable and observed two peaks with different loca¬ 
tions on the scalp. The first peak occurred around 
220 ms after stimulus onset and did not habituate to 
repeated presentations (except after the first presenta¬ 
tion) or dishabituate to the novel syllable. Thus, the 
brain generators of this peak, probably primary and 
secondary auditory areas in the temporal lobe, did not 
appear to be sensitive to the subtle acoustical differ¬ 
ences that encoded phonetic information. 

The second peak reached its maximum around 
390 ms after stimulus onset and again did not habitu¬ 
ate to repetitions of the same syllable, except after the 
first presentation. However, when the deviant sylla¬ 
ble was introduced, the peak recovered to at least its 
original level. Thus, the neural generators of the sec¬ 
ond peak, also in the temporal lobe, but in a distinct 
and more posterior location, are sensitive to phonetic 
information. Further studies need to be carried out 
to ascertain whether the recovery of the second peak 
is due to the categorical perception of phonemes, or 
whether it would be elicited by any change in sound. 

Further investigations by Dehaene-Lambertz and 
colleagues used two imaging methods with better spa¬ 
tial resolution to investigate early correlates of speech 
perception. For example, Dehaene-Lambertz et al. (2006) 
measured brain activation for forward and backward 
speech in 3-month-old infants. They found that forward 
and backward speech activated differing - although 
closely situated - brain areas in left temporal language 
areas (Figure 15.21). They also found that the forward 
speech activated right prefrontal cortex while the back¬ 
ward speech did not. Results provided evidence that as 
early as 3 months, infants show a left lateralization for 
speech that also activates frontal lobe regions. 

The general conclusions from the above stud¬ 
ies are reinforced by converging results from a new 
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methodology - near infrared spectroscopy (NIRS). In 
one experiment, Mehler and colleagues (Pena et ah, 
2003) played normal infant-directed speech or the 
same utterances played in reverse while they meas¬ 
ured changes in the concentration of total hemoglobin 
within parts of the right and left hemisphere. They 
observed that left temporal areas showed significantly 
more activation when infants were exposed to normal 
speech than to backward speech or silence, leading 
them to conclude that neonates are born with a left 
hemisphere already biased for speech processing. 

Language acquisition and speech perception have 
been some of the most active areas of developmental 
cognitive neuroscience. The use of converging meth¬ 
odologies, and frequent comparisons between typical 
and atypical trajectories of development, make it the 
domain most likely to see major breakthroughs over 
the next decade (see Gage et ah , 2003a, b for a dis¬ 
cussion of auditory mechanisms underlying speech 
development in children with autism disorder). 

4 A .2 Developing the executive brain: what do 
babies know? 

A critical aspect of an infant's cognitive growth dur¬ 
ing the first year of life is the ability to learn about his 
or her environment. New items will attract the atten¬ 
tion of a young infant and he or she will gaze at these 
items for longer durations than for items that are 
accustomed to being seen. While it is important for an 


infant to gaze at a new item, it is also important for an 
infant to orient to other aspects of his or her environ¬ 
ment. The trading effects of looking at new items and 
shifting attention to other elements in the world about 
them provides infants both with learning opportuni¬ 
ties for understanding features in new items and a 
wide range of such experiences by changing the focus 
of their attention, both of which are critical to cogni¬ 
tive development. 

The executive control for directing attention to new 
items, orienting, maintaining goals, and control over 
reaching movements are thought to require the most 
anterior portion of the brain, the prefrontal cortex 
(PFC) (see Chapter 12). As discussed earlier, the fron¬ 
tal cortex shows the most prolonged period of post¬ 
natal development of any region of the human brain, 
with neuroanatomical changes evident even into the 
teenage years (Huttenlocher, 1990; Giedd et ah, 1999). 
For this reason, it has been the part of the brain most 
commonly associated with developments in cognitive 
abilities during childhood. 

One of the most comprehensive attempts to relate 
a cognitive change to underlying brain developments 
has concerned the emergence of object permanence 
in infants. Object permanence is the ability to retain 
an object in mind after it has been hidden by another 
object or a cover (Figure 15.22). Specifically, Piaget 
observed that infants younger than around 7 months 
fail accurately to retrieve a hidden object after a short 
delay period if the object's location is changed from 



(b) 



Backward speech Forward speech Forward-Backward 


FIGURE 15.21 As early as 3 months, infants show 
a leftward laterality for human speech. Moving from 
left to right, the brain figures reflect (i) brain activation 
in the temporal lobe for backward speech (orange); (ii) 
brain response in a slightly higher brain slice, showing 
activation for both backward speech and forward speech 
(blue); (iii) brain response in a slightly higher brain splice 
showing activation for forward speech; (iv) the differing 
in brain activation for forward-backward speech shows 
activation in right prefrontal cortex. Source : Dehaene- 
Lambertz, et al, 2006. 
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FIGURE 15.22 Behavioral testing in infants, (a) An object retrieval task that infants fail up to 9 months of age. In full view of the infant, 
the experimenter hides the object in one location and the infant reaches for it successfully. After a few such trials, the experimenter hides the 
object in a second place but the infant searches again at the original location (Piaget, 1954). (b) A visual habituation technique can be used to 
show that infants from as young as 4 months perceive the left-hand figure as a continuous rod moving behind an occluder. Infants dishabitu¬ 
ated (found novel) the test display with two short rods, indicating that they perceptually Tilled in' the occluded area in the habituation dis¬ 
play Infants under 4 months are only partially successful in such tasks, depending on the complexity of the display, (c) The infant views two 
event sequences, one possible and one impossible, in which a flap is rotated toward a solid cube. In the "possible' case the flap stops when it 
comes into contact with the object. In the impossible case the flap rotates through the object. Infants as young as 4 months appear surprised 
(look longer) when viewing the impossible event, showing that they appreciate that objects are solid and (usually) non-compressible. Source : 
Johnson, 2001. 


one where it was previously and successfully retrieved. 
In particular, infants at this age make a particular per- 
severative error in which they persistently reach to 
the hiding location where the object was found on the 
immediately preceding trial. This characteristic pattern 
of error, called 'A not B', was cited by Piaget (1954) as 
evidence for the failure of infants to understand that 
objects retain their existence or permanence when 
moved from view. Beyond about 7 months infants 
begin to succeed in the task at successively longer 
delays of 1 to 5 seconds (Diamond, 1985,2001). 

Diamond and Goldman-Rakic (1989) found that 
infant monkeys also make errors in an adapted version 


of Piaget's object permanence task. Similar errors 
were also seen in adult monkeys with damage to dor¬ 
solateral prefrontal cortex (DL-PFC). Damage to other 
parts of cortex did not have the same effects, indicat¬ 
ing a specific role for DL-PFC in this task. 

Evidence linking this change in behavior to brain 
development also comes from EEG studies with 
human infants (Fox and Bell, 1990; Bell, 1992a, b; Bell 
and Fox, 1992). In these studies, increases in frontal 
EEG responses correlate with the ability to respond 
successfully over longer delays in delayed response 
tasks. Most recently, an optical imaging study with 
infants has revealed a correlation between behavioral 
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success at the AB task and blood oxygenation in pre¬ 
frontal cortex (Baird et al., 2002). 

Converging evidence for the importance of fron¬ 
tal cortex development comes from studies on chil¬ 
dren with an atypical neurochemical balance in the 
prefrontal cortex resulting from phenylketonuria 
(PKU) (Welsh et al., 1990; Diamond, 2001). Even when 
treated, this metabolic disorder can have the specific 
consequence of reducing the levels of a neurotrans¬ 
mitter, dopamine, in the dorsolateral prefrontal cortex. 
These reductions in dopamine levels in the dorso¬ 
lateral prefrontal cortex result in these infants and 
children being impaired on tasks thought to involve 
prefrontal cortex such as the object permanence task, 
and having typical performance in tasks thought to be 
dependent on other regions of cortex, such as delayed 
non-matching to sample (Welsh et al., 1990; Diamond, 
2001 ). 

Thus, converging evidence from several sources 
supports the view that development of DL-PFC allows 
infants to succeed in the object permanence task. 
According to Diamond (1991), the critical features of 
the task carried out by DL-PFC is the ability to retain 
information over spatial delays and to inhibit prepotent 
(previously reinforced) responses. However, two recent 
lines of evidence suggest that the DL-PFC develop¬ 
ment hypothesis is not the whole story and that some 
modification or elaboration of the original account 
may be required. The first of these lines of evidence 
comes from another task thought to require DL-PFC: 
the oculomotor delayed response task (Figure 15.23). 
Gilmore and Johnson (1995) found that infants can 
succeed on this task at a much younger age than is 
indicated by the object retrieval tasks, even though it 
also requires the infant to maintain spatial information 
over a delay and needs to inhibit a prepotent response. 

One explanation of the discrepancy between per¬ 
formance in Piaget's object permanence task and the 
results of Gilmore and Johnson (1995) is that tasks 
in which the response is an eye movement (looking) 
are easier since eye movement planning develops 
more rapidly than other forms of motor output such 
as reaching. This proposal is consistent with several 
studies showing that infants can perform successfully 
in analogous object permanence tasks as young as 
4 or 5 months of age if looking rather than reaching 
performance is measured (Lecuyer et al., 1992). 

In order to explain the difference between results 
obtained with eye movements as the measure and 
those with the manual reaching measures, it has been 
argued that infants do not yet have the necessary action 
planning skills to coordinate the sequence of motor 



FIGURE 15.23 The oculomotor delayed response task as 
designed for use with infants. Infant subjects face three computer 
screens on which brightly colored moving stimuli appear. At the 
start of each trial a fixation stimulus appears on the central screen. 
Once the infant is looking at this stimulus, a cue is briefly flashed up 
on one of the two side screens. Following the briefly flashed cue, the 
central stimulus stays on for between 1 and 5 seconds, before presen¬ 
tation of two targets on the side screens. By measuring delayed looks 
to the cued location prior to the target onset, Gilmore and Johnson 
(1995) established that infants can retain information about the cued 
location for several seconds. Source: Gilmore and Johnson, 1995. 

behaviors to retrieve a hidden object (Diamond, 1991; 
Baillargeon, 1993). To test this idea, Munakata et al. 
(1994) trained 7-month-old infants to retrieve objects 
placed at a distance from them by means of pulling on 
a towel or pressing a button. Infants retrieved objects 
when a transparent screen was interposed between 
them and the toy, but not if the screen was sufficiently 
opaque to make the object invisible. Since the same 
means-ends planning is required whether the screen 
is transparent or opaque, it was concluded that this 
cannot account for the discrepancy between the look¬ 
ing and the reaching tasks. Munakata et al. (1994) pro¬ 
posed an alternative 'graded' view of the discrepancy 
implemented as a neural network model. This model 
illustrates how weak internal representations of a stim¬ 
ulus can be sufficient to drive a simple output, such as 
an eye movement, but may be insufficient to initiate a 
more complex motor output, such as reaching. 

An alternative approach to understanding the 
role of the prefrontal cortex in cognitive develop¬ 
ment has been advanced by several authors who have 
suggested that the region plays a critical role in the 
acquisition of new information and tasks. From this 
perspective, the challenge to the infant brain in, for 
example, learning to reach for an object, is equivalent 
in some respects to that of the adult brain when fac¬ 
ing complex motor skills like learning to drive a car. 
Three predictions from this view are that (i) the cortical 
regions crucial for a particular task will change with 
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the stage of acquisition; (ii) the prefrontal cortex plays 
a role in organizing or allocating information to other 
regions of cortex; and (iii) that development involves 
the establishment of hierarchical control structures, 
with frontal cortex maintaining the currently highest 
level of control. Recent evidence showing PFC acti¬ 
vation early in infancy has given further credence to 
this view. The limited number of fMRI and PET stud¬ 
ies that have been done with infants have often sur¬ 
prisingly revealed functional activation in PFC, even 
when this would not be predicted from adult studies. 
For example, in an fMRI study of speech perception 
in 3-month-olds, Dehaene-Lambertz and colleagues 
(Dehaene-Lambertz et al., 2006) observed a right DL- 
PFC activation that discriminated (forward) speech in 
awake, but not sleeping, infants (Figure 15.21). Similar 
activation of DL-PFC was found in response to faces at 
the same age (Tzourio-Mazoyer et al., 2002). While this 
is evidence for activation of at least some of the PFC in 
the first few months, it remains possible that this acti¬ 
vation is passive as it does not play any role in direct¬ 
ing the behavior of the infant. 

Developmental ERP studies have often recorded 
activity changes over frontal leads in infants and some 
experiments suggest that this activity has important 
consequences for behavioral output. These experi¬ 
ments involve examining patterns of activation that 
precede the onset of a saccade. In one example, Csibra 
and colleagues (Csibra et al., 1998, 2001) observed that 
pre-saccadic potentials that are usually recorded over 
more posterior scalp sites in adults are observed in 
frontal channels in 6-month-old infants. Since these 
potentials are time-locked to the onset of an action, it 
is reasonable to infer that they are the consequence of 
computations necessary for the planning or execution 
of the action. 

Further evidence for the developmental importance 
of the PFC from early infancy comes from studies of 
the long-term and widespread effects of perinatal 
damage to PFC. Selective perinatal damage to the rel¬ 
evant regions activated in adults often has, at worst, 
mild effects on infants with subsequently nearly com¬ 
plete recovery of function. In contrast, perinatal dam¬ 
age to frontal and PFC regions often results in both 
immediate and long-term difficulties. This generali¬ 
zation from several domains suggests that PFC could 
play an important structuring or enabling role from 
very early in postnatal development. 

The issue raised at the beginning of this section 
concerned how to reconcile evidence for continuing 
neuroanatomical development in the frontal cortex 
until the teenage years on the one hand, and evidence 


for some functioning in the region as early as the first 
few months of age on the other. One possible resolu¬ 
tion to this issue is that representations that emerge 
within this region of cortex are initially weak and suf¬ 
ficient only to control some types of output, such as 
saccades, but not others, such as reaching (Munakata 
et al., 1994). Other plausible resolutions of this issue 
come from Diamond's (1991) proposal that differ¬ 
ent regions of frontal cortex are differentially delayed 
in their development, and Thatcher's (1992) suggestion 
that prefrontal regions may have a continuing role in 
the cyclical reorganization of the rest of cortex. 

Whether these hypotheses work out or not, there is 
good reason why some degree of PFC functioning is 
vital from the first weeks of postnatal life, or even ear¬ 
lier (Fulford et al., 2003). The ability to form and retain 
goals, albeit for short periods, is essential for gener¬ 
ating efforts to perform actions such as reaching for 
objects. Early and often initially unsuccessful attempts 
to perform motor actions provide the essential experi¬ 
ence necessary for subsequent development. 

4 . 1.3 Developing the social brain: faces and places 

As described in Chapter 14, one of the major charac¬ 
teristics of the human brain is its social nature. A vari¬ 
ety of cortical areas have been implicated in the 'social 
brain' including the superior temporal sulcus (STS), 
the fusiform 'face area' (FFA) and orbitofrontal cortex. 
One of the major debates in cognitive neuroscience 
concerns the origins of the 'social brain' in humans 
and theoretical arguments abound about the extent to 
which this is acquired through experience. 

One aspect of social brain function in humans that 
has been the topic of many investigations is the per¬ 
ception and processing of faces. There is a long history 
of research on the development of face recognition in 
young infants extending back to the studies of Fantz 
more than 40 years ago (e.g. Fantz, 1964). Over the 
past decade, numerous papers have addressed the cor¬ 
tical bases of face processing in adults, including iden¬ 
tifying areas that may be specifically dedicated to this 
purpose (see Chapter 6). Despite these bodies of data, 
surprisingly little remains known about the develop¬ 
mental cognitive neuroscience of face processing. 

In a review of the available literature in the late 1980s, 
Johnson and Morton (1991) revealed two apparently 
contradictory bodies of evidence: while the prevail¬ 
ing view, and most of the evidence, supported the idea 
that infants gradually learn about the arrangement of 
features that compose a face over the first few months 
of life, the results from at least one study indicated that 
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BOX 15.1 Face preferences in newborns 


Over the past decade more than a dozen papers have 
been published on the face-related looking preferences 
of newborn infants. Most of these papers concluded that 
newborns are biased to attend to stimuli that possess 
certain characteristics of faces, but two alternative views 
have been expressed. 

The first of these views (the 'sensory hypothesis') is that 
all newborn visual preferences, including those for face- 
related stimuli, can be accounted for simply in terms of 
the relative visibility of the stimuli. The newborn visual 
system is restricted to the lower part of the range of spatial 
frequencies that is visible to adults. It has been proposed 
that newborns prefer to look at faces merely because the 
amplitude at different frequencies of these stimuli hap¬ 
pens to best match the sensitivity of the newborn visual 
system. The sensory hypothesis has fallen out of favor 
because, even when amplitude is controlled, phase infor¬ 
mation (configuration) still influences the newborn pref¬ 
erence toward faces. In addition, attempts to simulate 
newborn preferences with neural network models based 


on the sensory hypothesis are unlikely to account for other 
experiments involving realistic faces within the complex 
visual scenes to which newborns are exposed. 

The second alternative view is that we have complex 
face processing abilities from birth. The findings used 
to support this claim include a preference for images of 
attractive faces, data indicating that newborns are sensitive 
to the presence of eyes in a face, and evidence that they 
prefer to look at faces that engage them in eye contact. 
In addition to the immaturity of the cortex at birth, all of 
these results could be accounted for by a low spatial fre¬ 
quency face configuration detector. For example, older 
infants prefer more attractive faces because these faces are 
closer to an average or prototypical face. Inspection of real¬ 
istic face images through the appropriate spatial frequency 
filters for newborns reveals that a mechanism that is sensi¬ 
tive to the configuration of a face could be preferentially 
activated by (i) the most prototypical face configuration 
presented, (ii) the presence (or absence) of open eyes, and 
(iii) direct versus averted gaze (see Figure 15.23). 


BOX 15.2 Why does the fusiform develop a face area? 


A central debate in cognitive neuroscience concerns the 
origin and specificity of the fusiform face area (FFA) and 
other face-sensitive regions of cortex. One view is that the 
FFA is selectively activated by faces owing to genetically 
specified and domain-specific computational properties 
of that region. In contrast, others have proposed that the 
region is involved in processing visual stimuli in domains 
of perceptual expertise and most human adults become 
experts in face processing. From the developmental per¬ 
spective taken in this chapter, an alternative 'middle way' 
account of FFA emerges. By this account, parts of the 
fusiform cortex become specialized for processing faces 
as a result of several constraining factors. First, the sub¬ 
cortical route described in this chapter ensures that new¬ 
borns preferentially orient to faces and therefore foveate 
them, thus providing input to cortical visual pathways. 
Second, the cortical projection patterns of the subcortical 


newborn infants, as young as 10 minutes old, will track 
a face-like pattern farther than various 'scrambled' face 
patterns (Goren et ah, 1975). Evidence that newborns 
showed a preferential response to faces was used by 
some to bolster nativist views of infant cognition. In 
contrast, the evidence for the graded development of 
face processing abilities over several months tended to 
be cited by theorists who believed that such skills need 
to be learned, and result from experience of the world. 

Although detailed issues remain, the finding that 
newborns preferentially respond to patterns that 


route may enhance activation of specific areas, includ¬ 
ing the fusiform cortex, when faces are within the visual 
field of the young infant. Third, the parts of the fusiform 
cortex that become face sensitive receive foveal corti¬ 
cal visual input and are at the 'object-level' of visual 
stimulus processing in the ventral pathway. Thus, infor¬ 
mation from both face routes may converge in the FFA. 
These and other possible constraints, such as multi¬ 
modal inputs and general biases in gene expression lev¬ 
els between the right and left cerebral cortex, combine to 
ensure that certain developing cortical circuits become 
specialized for face-related stimuli. By this developmen¬ 
tal account it is inevitable, barring some disruption to the 
normal constraints, that parts of the fusiform cortex will 
specialize for faces. However, this inevitable outcome is 
achieved without genetically specified domain-specific 
patterns of connectivity with the FFA. 


correspond to faces remains valid (see Johnson, 2005b, 
for review) (Figure 15.24). Recent evidence from a vari¬ 
ety of sources indicates that there may be a subcortical 
route for face processing involving the superior collicu¬ 
lus, pulvinar, and amygdala (Johnson, 2005b). While this 
route is only evident in adults with functional imaging, 
or after acquired brain damage, it may be the main influ¬ 
ence over the behavior of the newborn in which cortical 
routes for visual processing are relatively immature. 

How may we investigate face perception in infants? 
Several laboratories have examined changes in 






490 


15. DEVELOPMENT 


50 n 



Face Scrambled Blank 


FIGURE 15.24 Face perception in newborns. 
Data showing the extent of newborns' head and 
eye turns in following a schematic face, a scrambled 
face, and a blank (unpatterned) stimulus. The infant 
tracked the face significantly farther than the other 
stimuli. Source : Johnson et al., 1991. 



FIGURE 15.25 Brain activation for faces (shown in red), buildings (shown in green) and objects (shown 
in blue) for children 5-8 years (left panel), adolescents 11-14 years (center panel) and adults (right panel). 
Results showed that children 5-8 years did not sow the face selectivity of older adolescents and adults. Source : 
Kadosh and Johnson, 2007. 


event-related potentials (ERPs) as adults view faces. 
In particular, interest has focused on an ERP compo¬ 
nent termed the 'N170' (because it is a negative-going 
deflection that occurs after around 170 ms) that has 
been strongly associated with face processing in a 


number of studies on adults. Specifically, the timing 
and size of this component vary according to whether 
or not faces are present in the visual field of the adult 
volunteer under study. An important aspect of the 
N170 in adults is that its response is highly selective. 
























4.0 DEVELOPING MIND AND BRAIN 


491 


For example, the N170 shows a different response to 
human upright faces than to very closely related stim¬ 
uli such as inverted human faces and upright monkey 
faces. While the exact underlying neural generators 
of the N170 are currently still debated, the specificity 
of response of the N170 can be taken as an index of 
the degree of specialization of cortical processing 
for human upright faces. For this reason de Haan, 
Johnson, and colleagues (2002) undertook a series of 
studies on the development of the N170 over the first 
weeks and months of postnatal life. 

The first issue addressed in these developmen¬ 
tal ERP studies is when does the face-sensitive N170 
emerge? In a series of experiments, a component 
was identified in the infant ERP that has many of the 
properties associated with the adult N170, but that is 
of a slightly longer latency (240-290 ms). In studying 
the response properties of this potential at 3, 6, and 
12 months of age it was discovered that (1) the compo¬ 
nent is present from at least 3 months of age (although 
its development continues into middle childhood), 
and (2) the component becomes more specifically 
tuned to human upright faces with increasing age. To 
expand on the second point, it was found that while 
12-month-old infants and adults showed different 
ERP responses to upright and inverted faces, 3- and 
6-month-old infants do not. Thus, the study of this 
face-sensitive ERP component is consistent with the 
idea of increased specialization of cortical processing 
with age, a result also consistent with recent func¬ 
tional neuroimaging studies. 

In a study by Sherf and colleagues (Sherf et al., m 
2007), played movie clips of faces, objects, and build¬ 
ings to children (5-8 years), adolescents (11-14 years) 
and adults. They found that the children did not show 
face specific brain activation (Figure 15.25). This find¬ 
ing provides evidence that although face processing 
may have an initial bias in neural processing, expe¬ 
rience plays a key role in the development of face- 
selective cortex (Kadosh & Johnson, 2007). 

Converging evidence about the increasing specializa¬ 
tion of face processing during development comes from 
a behavioral study that set out to test the intriguing idea 
that, as processing 'narrows' to human faces, infants 
will lose their ability to discriminate non-human faces. 
Pascalis and colleagues (Pascalis et al., 2002) demon¬ 
strated that while 6-month-old infants could discrimi¬ 
nate between individual monkey faces as well as human 
faces, 9-month-old infants and adults could only dis¬ 
criminate the human faces. These results are particularly 
compelling since they demonstrate a predicted compe¬ 
tence in young infants that is not evident in adults. 


Moving beyond the relatively simple perception 
of faces, a more complex attribute of the adult social 
brain is processing information about the eyes of other 
humans. There are two important aspects of process¬ 
ing information about the eyes. The first of these is 
being able to detect the direction of another's gaze in 
order to direct your own attention to the same object 
or spatial location. Perception of averted gaze can elicit 
an automatic shift of attention in the same direction 
in adults, allowing the establishment of 'joint atten¬ 
tion'. Joint attention to objects is thought to be crucial 
for a number of aspects of cognitive and social devel¬ 
opment, including word learning. The second criti¬ 
cal aspect of gaze perception is the detection of direct 
gaze, enabling mutual gaze with the viewer. Mutual 
gaze (eye contact) provides the main mode of estab¬ 
lishing a communicative context between humans and 
is believed to be important for normal social develop¬ 
ment. It is commonly agreed that eye gaze perception 
is important for mother-infant interaction and that it 
provides a vital foundation for social development. 

In a series of experiments with 4-month-old infants 
using a simple eye gaze cueing paradigm, Farroni and 
colleagues (Farroni et al., 2000) have established that 
it is only following a period of mutual gaze with an 
upright face that cueing effects are observed. In other 
words, mutual gaze with an upright face may engage 
mechanisms of attention such that the viewing infant 
is more likely to be cued by subsequent motion. In 
summary, the critical features for eye gaze cueing in 
young infants are (1) lateral motion of elements and 
(2) a brief preceding period of eye contact with an 
upright face. 

Following the surprising observation that a period of 
direct gaze is required before cueing can be effective in 
infants, the earliest developmental roots of eye contact 
detection have been investigated. It is already known 
that human newborns have a bias to orient toward face¬ 
like stimuli (see earlier), prefer faces with eyes opened, 
and tend to imitate certain facial gestures. Preferential 
attention to faces with direct gaze would provide the 
most compelling evidence to date that human new¬ 
borns are born prepared to detect socially relevant 
information. For this reason we investigated eye gaze 
detection in humans from birth. Farroni and colleagues 
tested healthy human newborn infants by presenting 
them with a pair of stimuli, one a face with eye gaze 
directed straight at the newborns and the other with 
averted gaze. Results showed that the fixation times 
were significantly longer for the face with the direct 
gaze. Further, the number of orientations was higher 
with the straight gaze than with the averted gaze. 
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In a second experiment, converging evidence for 
the differential processing of direct gaze in infants was 
obtained by recording event-related potentials (ERPs) 
from the scalp as infants viewed faces. Farroni and col¬ 
leagues studied 4-month-old babies with the same 
stimuli as those used in the previous experiment with 
newborns and found a difference between the two gaze 
directions at the time and scalp location at the previously 
identified face-sensitive component of the infant ERP 
discussed earlier. The conclusion from these studies is 
that direct eye contact enhances the perceptual process¬ 
ing of faces even in infants as young as 4 months. 

Beyond face processing and eye gaze detection, 
there are many more complex aspects of the social 
brain, such as the coherent perception of human action 
and the appropriate attribution of intentions and goals 
to conspecifics. Investigating the cognitive neuro¬ 
science of these abilities in infants and children will 
be a challenge for the next decade. One way in which 
these issues have been addressed is through study¬ 
ing genetic developmental disorders in which aspects 
of social cognition are either apparently selectively 
impaired (autism) or, selectively intact amid otherwise 
impaired cognition (Williams syndrome). 

The first year of life brings dynamic changes in both 
behavior and the brain. While we provided separate 
discussions of the emergence of language, executive 
functions, and social cognition in this busy first year 
of life, these three domains of human cognition have 
complex interactions throughout development. An 
infant's interest in faces, for example, will help him or 
her to understand speech. The ability to focus on new 
objects aids the infant in developing knowledge about 
the world around him or her. While we do not fully 
understand the explosive growth of brain processes 
and their relation to behavior in infants, studies such 
as the ones we presented here are helping to map the 
complex correspondence between mind and brain. 

4.2 Childhood and adolescence: dynamic and 
staged growth 

While the first year of life represents an unparalleled 
stage in dynamic human development, many aspects 
of brain and cognitive growth take years to mature. As 
we mentioned earlier, the field of developmental cog¬ 
nitive neuroscience is a young one. In this section, we 
will present results from some studies investigating the 
development of brain areas subserving language, exec¬ 
utive functions, and social cognition in children and 
adolescents and its relation to behavior. While these 
studies are informative, it is important to bear in mind 


that relating complex brain activation to performance 
on cognitive tasks is a highly complex process. For 
example, it may well be the case that the brain activity 
that we observe in adults - once a cognitive process has 
been developed - may tap differing brain regions while 
it is being acquired. Thus, simply comparing regions of 
interest in neuroimaging experiments across groups 
of children versus adults may not provide us with the 
level of sensitivity that we require in order to formulate 
inferences about brain and behavior. Similarly, differ¬ 
ing cognitive strategies or coping mechanisms in child¬ 
hood versus adulthood may also impact the network of 
brain areas tapped in certain task paradigms. 

With those caveats in mind, let's review the evi¬ 
dence regarding the development of neural systems 
for language, executive control, and social cognition. 

4 . 2 .1 The linguistic brain: language acquisition 

Language is not a unitary system: in order to express 
our ideas verbally, we need to progress through 
stages of formulating the concepts, mapping them 
onto words in our mental lexicon, accessing our men¬ 
tal grammar to form sentences, and mapping this 
information onto sound-based representations of the 
articulation of the ideas we want to express. Thus, 
the language system has multiple stages of computa¬ 
tion across many categories or aspects of language. 
It makes intuitive sense that, early in life, infants 
develop their knowledge about language based largely 
on the sounds of language that they hear in their envi¬ 
ronment. Thus, it is not surprising that studies with 
young infants (less than 12 months) typically focus on 
the phonology of human language. With older infants, 
children, and adolescents, studies typically test other 
aspects of language such as lexical-semantic (meaning 
based) and syntactic (grammar based) knowledge. 

Do all aspects of language develop in similar ways, 
with similar brain developmental processes underly¬ 
ing them? This is a question that has been addressed 
by developmental cognitive neuroscientists investigat¬ 
ing the neural substrates of human language. Here is a 
brief summary of what we have learned so far. 

We have previously discussed the use of event- 
related potentials (ERPs) for measuring the time 
course of human brain response for stimulus events. 
Several ERP components have been used to investi¬ 
gate language emergence in infants and young chil¬ 
dren and we describe them here: 

• The mismatch negativity (MMN) component: 
this component shows a negativity at 100-250 ms 



4.0 DEVELOPING MIND AND BRAIN 


493 


C/) 

0 

CD 

ro 

to 

Identification of word 
boundaries 

76 




c 




0 




E 

CL 

o 

0 

s 

D 

Discrimination 
of phonemes 


Discrimination 
word stress 
pattern 


s ■- 

I-1- 

Birth 2 


□_ 

□C 

LU 


MMN 


Identification of 
intonational 
boundaries 


Lexical 

processing 





Sensitivity to 
intonational 


Lexical 


Lexical 

phrase boundary 


form 


semantics 


i ' iV 

f-MM—I- 

5 6 8 9 12 14 18 


Sentence processing 


Selectional 
restriction 
of verbs 


Local 

phrase 

structure 

building 


Morpho- 

syntactic 

processes 



24 30 32 36 

Months of age 


CPS 


N400 


ELAN - P600 


-► 


FIGURE 15.26 A schematic overview of the developmental stages of auditory language perception and the ERP correlates that provide 
the possibility to investigate phonological, semantic and syntactic processes. The developmental stages can be viewed as interrelated steps 
during which novel information is extracted and processed on the basis of previously acquired knowledge. Once the basic phonological 
processes are established, phonemic knowledge is used to identify and represent the first lexical forms and create a larger lexical semantic 
knowledge base, which is then used to process meaning in sentential context. The depicted time course of the different developmental stage 
is an approximation and is based on the ERP studies available in the literature. This also holds for the relation between the developmental 
age and the ERP components reported in the different studies discussed in the text. Source : Friederici, 2005. 


which reflects the ability for the brain to discrimi¬ 
nate between acoustic/phonetic features in sounds 

• The N400, a negativity at —400 ms which has 
source generators in centro-parietal regions. The 
N400 is thought to reflect lexical-semantic pro¬ 
cesses in word and sentence comprehension 

• A left anterior negativity (E/LAN) which occurs 
—150-350ms and reflects online syntactic processes 

• The P600, a positive deflection at —600 ms with a 
centro-parietal distribution of sources. The P600 
reflects processes engaged in syntactic revision and 
reanalyses. 

Friederici (2005) describes the time course of lan¬ 
guage development in infants and children in a recent 
review of the child language literature (Figure 15.26). 
According to this review, phonological and intonational 
processes develop relatively early (birth to 9 months) 
while lexical, semantic, and syntactic processes develop 
somewhat later (1-3 years) as infants and young chil¬ 
dren acquire information at word and sentence levels. 

During the first 2 years of life, there is an explo¬ 
sion of language knowledge and abilities as the infant 
begins to babble, produce simple sounds ('mama'), 
words ('dog'), and two- to three- word utterances 


('want juice'). By age 2, a child can typically produce 
many words. But what does a 2-year-old understand 
about the meaning of words? Are they simply mim¬ 
icking words /sounds they have heard? Or do they 
have knowledge beyond simple repetition of sound? 
Researchers have devised clever experimental proce¬ 
dures to investigate the semantic knowledge of infants 
and young children. One method is to present a picture 
of an object that the child is familiar with, for exam¬ 
ple, a duck, following by the word 'duck' (congruous 
condition) or a word that does not match the picture 
('cat', incongruous condition). These careful testing 
procedures have enabled language researchers to test 
very young children in order to elucidate their level of 
semantic knowledge. Results of one such study are pre¬ 
sented in Figure 15.27, providing evidence that infants 
as young as 14 months have semantic contextual 
knowledge, reflected in N400 responses for congruous 
and incongruous events (Friederici, 2005). 

What do 2-years-olds know about the grammati¬ 
cal rules of their language? Sentence level process¬ 
ing was investigated in a sample of 2-years-olds and 
adults by contrasting sentences that were grammati¬ 
cally well formed ('The lion roars') to sentences that 
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FIGURE 15.27 Semantic processing to congruous and incongruous words. The brain response 
of (a) 14 months old, (b) 19 months old, and (c) adults in a picture priming paradigm. A picture was 
presented for 4 s on a screen and 900 ms after the picture onset an indefinite article was acoustically 
presented, followed Is later by a word that was either congruous with the picture (e.g. a picture 
of a duck followed by the word 'duck') or not (a picture of a duck followed by the word "cat'). The 
solid line represents the ERP to the congruous word condition and the broken line to the incongru¬ 
ous word condition. ERPs shown for one selected electrode (Z) - a negativity starting around 400 ms 
(N400) is observable in all age groups, ((a) Adapted with permission from Friedrich and Friederici, 
2005. (b,c) adapted from Friedrich and Friederici, 2005). Source : Friederici, 2005. 


2-year-olds Adults 



FIGURE 15.28 Syntactic processes in 2-year-olds, (a) Brain responses of 2-year-old children and adults to phrase 
structure violations in simple active sentences. ERPs are shown for the critical last word in the syntactically correct 
condition (solid line) and the syntactically incorrect condition (broken line), from the word onset for two selected elec¬ 
trodes (F7, PZ). An early left anterior negativity (ELAN) and a late positivity (P600), both known to be elicited in 
adults in response to phrase structure errors, were also observed for 2-year-old children, (b) Topographic maps show¬ 
ing the difference in brain activation when the correct condition is subtracted from the incorrect condition, indicating 
the topographical distribution of the effects in 2-year-olds (left) and adults (right). (Adapted from Oberecker et al., 
2005). Source : Friederici, 2005. 


were not grammatical (The lion in the roars')- The 
2-years-olds were able to discriminate between well- 
formed and ill-formed sentences, as indexed by the E/ 
LAN and P600 components (Figure 15.28). 


How do these ERP studies inform us about brain 
processes for language acquisition in childhood? 
While these types of studies are in their early stages, 
these investigations reflect developments in ERP 
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recording techniques and experimental designs that 
are effective at providing sensitive measures of early 
language knowledge. While more work is needed to 
understand fully the complexities of human language 
acquisition, these studies provide important new data 
about language knowledge in very young children. 

Cortical brain areas develop throughout childhood 
and adolescence, as mentioned in earlier sections of this 
chapter. How do brain regions for speech perception 
and production develop and mature in childhood? 
Classical brain areas for language include a frontal 
lobe region (functionally identified as Broca's area) 
and a temporal lobe region (functionally known as 
Wernicke's area) that subserve speech production and 
perceptual processes. Throughout childhood, these 



(a) (b) 


FIGURE 15.29 Age related changes in white matter in two 
regions in the brain. An increase in white matter density was shown 
for the internal capsule (left panel) and the arcuate fascinulus in the 
left hemisphere (right panel) in a sample of 111 children age 4—17 
years. (Reprinted with permission from Paus et al., 1999.) Source : Paus 
et al., 2005. 


regions become connected through the experiences of 
producing and perceiving auditory language. A study 
by Paus and colleagues (Paus et al., 1999) used fMRI to 
investigate the development of connections between 
Broca's area and Wernicke's area using white matter 
density measures in a large sample (N = 111) of chil¬ 
dren in the age range 4-11 years. Their results show age- 
related changes in the internal capsule (Figure 15.29, 
left panel) and the left arcuate fasciculus, the fiber tract 
thought to connect Broca's and Wernicke's areas in the 
left hemisphere (Figure 15.29, right panel). 

Recall that measures of cortical thickness have been 
used to investigate patterns of development in cortex. 
Are there specific changes in anatomical brain regions 
that support language function? Toga and colleagues 
(Toga et al ., 2005) investigated this using a longitudi¬ 
nal approach in a sample of 45 typically developing 
children studied between the ages of 5 and 11 years. 
Cortical thickening increases of —0.10-0.15 mm/year 
were found in brain regions corresponding to Broca's 
and Wernicke's areas (Figure 15.30). 

Does this evidence for cortical thickening in brain 
regions subserving language reflect the ongoing and 
dynamic changes in child language? It is likely the case, 
but more studies relating these anatomical changes to 
language ability must occur before we fully understand 
the mind-brain correspondence in the development of 
language. 

In this section, we have highlighted evidence that 
very young children have sophisticated knowledge 
about semantic and syntactic information in spoken lan¬ 
guage. We have also shown that language regions in the 
brain continue to develop throughout childhood and 
into adolescence. However, there are many questions 
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FIGURE 15.30 Annualized rate of change in cortical thickness. The average rate of 
change in cortical thickness is shown in millimeters according to the color bar on the right 
(maximum gray matter loss is shown in shades of red and maximum gray matter gain is shown 
in shades of blue). Forty-five children were studied twice (two-year scan interval) between 
5 and 11 years of age. Source : Toga et al., 2005, from Sowell et al., 2004, with permission. 
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that remain unanswered regarding language acquisi¬ 
tion. A central issue that remains unresolved is the trad¬ 
ing relationship between nature (genetic predisposition 
for language) and nurture (the role of experience). Other 
aspects of language acquisition that are currently under 
investigation are the development of language systems 
in children who are bi lingual or multi lingual. 

4.2.2 The executive brain: taking cognitive control 

Even young infants must learn what information in their 
world is important and what is unimportant or irrele¬ 
vant. These learning mechanisms fall under the general 
category of 'cognitive control' and have been the focus 
of much study in infant and child development. Recall 
that, in infants, the A not B task has been used to inves¬ 
tigate the ability to ignore or inhibit irrelevant informa¬ 
tion and to inhibit prepotent response (Piaget, 1937, 
1954; Diamond, 1985). These capabilities become more 
and more important throughout childhood as a child's 
environment becomes increasingly complex. Consider 
a 6-year-old child in a first grade classroom: this child 
must be able to attend to the teacher or to a task at hand 
despite the many distractions that surround him, such 
as children talking, books dropping, chairs scraping. 
The trading of attentional resources toward relevant 
aspects of the environment and away from less impor¬ 
tant aspects is a vital element in development. 

In adults, the DL-PFC is implicated as an impor¬ 
tant cortical region in tasks that tap cognitive control 
functions. We know from histological and neuroana- 
tomical studies of developing children that the PFC 
has a prolonged developmental path, not reaching 
mature, adult-like stages until mid- to late adoles¬ 
cence. Behavioral studies of cognitive control function 
in children and adolescents have provided evidence 
for a similar time course in the development of cogni¬ 
tive control abilities. An open question in the field of 
developmental cognitive neuroscience is there corre¬ 
spondence between these late-to-mature brain regions 
and the late developing cognitive control abilities? 

Studies of the neural substrates of cognitive con¬ 
trol have only recently been undertaken with chil¬ 
dren. Casey and colleagues (see Casey et ah, 2005, for 
a review) have conducted seminal studies of cognitive 
control using a combination of fMRI and behavioral 
methods in order to investigate the neural patterns 
of brain activation measured while children perform 
tasks likely to engage PFC regions. In one experi¬ 
ment, they used event-related fMRI while children 
and adults were engaged in a 'go-no go' task (Durston 
et al., 2002). In this task, participants had to suppress 


their response when presented with a particular visual 
item within an ongoing sequence of stimulus presen¬ 
tations (e.g. one Pokemon character within a sequence 
of other Pokemon characters). The difficulty of the 
task was increased by increasing the number of 'go' 
items that preceded the 'no go' character. Successful 
response inhibition was associated with stronger acti¬ 
vation of prefrontal regions for children than for adults. 
Also, while in adults the activation of some prefrontal 
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FIGURE 15.31 The development of human cortical function, 
as measured by contemporary imaging methods, reflects fine-tuning 
of a diffuse network of neuroanatomical regions. Collectively, devel¬ 
opmental neuroimaging studies of cognitive control processes sug¬ 
gest a general pattern of increased recruitment of slow maturing 
prefrontal cortex (references depicted here in red), especially dorso¬ 
lateral prefrontal cortex and ventral prefrontal cortex, and decreased 
recruitment of lower level sensory regions (references in blue), 
including extrastriate and fusiform cortex and also posterior pari¬ 
etal areas. Importantly, specific activations vary with task demands, 
so working memory and Stroop tasks recruit different regions from 
response inhibition tasks. This pattern of activity, which has been 
observed across a variety of paradigms, suggests that higher cog¬ 
nitive abilities supported by association cortex become more focal 
or fine-tuned with development, whereas other regions not specifi¬ 
cally correlated with that specific cognitive ability become attenu¬ 
ated. A = anterior; P = posterior. Source : Casey et al ., 2005. 
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regions increased with increasing numbers of preced¬ 
ing 'go' trials (consistent with increasing need for 
inhibition), in children the circuit appeared to be maxi¬ 
mally active for all trial types. Along with the poorer 
behavioral performance of children in this and other 
inhibitory tasks, these findings suggest that the func¬ 
tional development of some PFC regions is important 
for the mature ability to inhibit prepotent tendencies. 

A general finding of Casey and colleagues has been 
that younger children exhibit broader, more diffuse 
brain activation for cognitive control tasks as compared 
to adults. During development, these brain areas mature 
and the brain activity that correlates to task performance 
abilities (such as reaction time and accuracy) become 
more focal and fine-tuned. In Figure 15.31, we present 
a figure from a recent article by Casey and colleagues 
(Casey et al, 2005) reviewing the literature of develop¬ 
mental cognitive control investigations. The general 
notion of brain activation becoming more focal and 


defined as a function of a child's age is shown with refer¬ 
ences to those studies showing increased activation with 
age and those showing decreased activation with age. 

Another approach to elucidate the role of frontal 
circuits in tasks that tap cognitive control functions 
was provided in a recent fMRI study of children aged 
8-12 years (Konrad et al., 2005). Konrad and colleagues 
utilized a well-accepted model for adult attentional 
processes (Posner and Petersen, 1990) and combined 
it with a task that has been well described in how it 
engages attentional processes, the Attention Network 
Task (ANT) (Fan et al, 2002) (Figure 15.32). 

The attention model proposed by Posner and col¬ 
leagues describes differing (separable but highly over¬ 
lapping) neural networks for: 

1 alerting to new and relevant information 

2 orienting and reorienting to relevant information 
or stimuli 



FIGURE 15.32 Experimental paradigm: modified version of the Attentional Network Task (Fan et al., 2002). This figure illustrates the 
time course of the four different cue and the two target conditions. Source : Konrad et al, 2005. 
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FIGURE 15.33 Alerting results. Differential activation of adults 
and children as identified in a two-sample test for the alerting condi¬ 
tion (thresholded at p svc <0.05, extend threshold 5 voxel, shown on 
averaged group T1 image), (a) Increased activation in the right cingu- 
lated gyrus in adults compared to children, (b) Increased brainstem 
activity in adults compared to children. Plots of the percent BOLD 
signal change are shown separately for both groups as a function of 
trial type (pooled over congruent and incongruent targets) for the 
respective activation maximum. Source : Konrad et al., 2005. 

3 executive control of attentional processes (Posner 
and Peterson, 1990). 

For example, consider the 6-year-old child in a first 
grade classroom. In order to cope with the complex 
environment with many sensory inputs compet¬ 
ing for his attention, he must first be ready and in an 
alert state. Next, he must be flexible in changing his 
attentional focus to orient to new sensory inputs, for 
example, or to reorient to relevant information when 
required. Last, he must have a means for controlling 
these attentional resources. 

These processes have been the focus of much study 
in adults. In adults, brain areas for alerting processes 
are located in frontal and parietal regions in the right 
hemisphere (Witte and Marrocco, 1997). Brain areas 
for orienting are located in the right hemisphere 
temporo-parietal junction and the inferior frontal 
gyrus (Corbetta et al., 2000). Brain areas for executive 


control include the anterior cingulate and lateral 
prefrontal cortex (Marrocco and Davidson, 1998). 
Konrad and colleagues (Konrad et al., 2005) used these 
brain areas that have been demonstrated to be active 
in adults during performance of the ANT as regions of 
interest in an fMRI study of 16 boys, aged 8-12 years, 
and 16 male adults. 

Results for the alerting portion of the protocol are 
presented in Figure 15.33. Children did not show 
adult-like activation patterns in the predefined regions 
of interest. The key activation differences between 
adults and children were in the right frontal cingulate 
gyrus (Figure 15.33, upper panel) and the midbrain 
(Figure 15.33, lower panel). The authors suggested 
that these differing patterns of activation are due to 
modulation by Top-down' mechanisms used in the 
task by the adults that may not be fully established in 
children in the age range of 8-12 years. 

Results for the reorienting portion of the protocol 
showed a typical pattern of right hemisphere activa¬ 
tion for the adults, while the children showed a more 
diffuse and broader distribution of activation during 
reorienting. These differences can be summarized as 
increased activation in the right side temporo-parietal 
junction in adults versus children (Figure 15.34(a)); 
stronger activation in the putamen and insula in chil¬ 
dren versus adults (Figure 15.34(b)); and increased 
activation in the superior frontal gyrus in children ver¬ 
sus adults (Figure 15.34(c)). These findings may rep¬ 
resent the recruitment of brain regions for performing 
the tasks due to immature cognitive control mecha¬ 
nisms in young children. 

The results of the executive attention portion of the 
protocol showed increased activation in the superior 
parietal cortex and inferior frontal gyrus in adults 
versus children (Figure 15.35(a,b)), and increased 
activation in the superior temporal gyrus and supe¬ 
rior frontal gyrus in children versus adults (Figure 
15.35(c,d)). 

Cumulatively the results of the study by Konrad 
and colleagues show a strikingly differing pattern of 
brain activation for children versus adults when per¬ 
forming attentional tasks. While these results must be 
treated with caution until more studies can be com¬ 
pleted using similar tasks and brain measures, the 
results of Konrad and colleagues provide evidence for 
immature frontal-parietal networks in cognitive con¬ 
trol tasks, a finding similar to the results reported by 
Casey and colleagues (reviewed in Casey et al., 2005). 

A general conclusion from these fMRI studies of 
cognitive control in children provides evidence for 
attentional systems that are functional but immature 









4.0 DEVELOPING MIND AND BRAIN 


499 


Reorienting 



(c) 



■ valid cue 

■ invalid cue 


adults children 

Group 



adults children 
Group 



adults children 


Group 


FIGURE 15.34 Differential activation of adults and children 
as identified in a two-sample test for the reorienting condition 
(thresholded at p svc <0.05 or p < 0.1 corrected for multiple com¬ 
parisons for whole-brain analyses, extend threshold 5 voxel, shown 
on averaged group T1 image), (a) Increased activation in the right¬ 
sided temporo-parietal junction in adults compared to children, 
(b) Stronger activation in the putamen and insula in children com¬ 
pared to adults, (c) Increased activation in the superior frontal gyrus 
in children compared to adults. Plots of the percent BOLD signal 
change are shown separately for both groups as a function of trial 
type (pooled over congruent and incongruent targets) for the acti¬ 
vation maximum. Source: Konrad et al., 2005. 


in children as compared to adults. Children show a 
pattern of activation that is more diffuse and encom¬ 
passes a wider area of brain regions than those active 
in adults. These findings may reflect the immature 
nature of the systems during childhood, which may 
become more focal and specialized with development 
and experience. 

Future work in the investigation of cognitive con¬ 
trol and development of PFC in children will need 
to take into account many other aspects of the pro¬ 
longed developmental path of the frontal regions and 
the correspondence to behavior. Some areas of future 
research may include investigating gender differ¬ 
ences in cognitive control functions and related brain 
activation patterns. You may have noticed that most 
of the tasks detailed in this section included visual 
presentation of task items. More work is needed to 
evaluate the frontal lobe networks that mediate audi¬ 
tory attentional control as well as those that subserve 
tasks that require integration of multimodal (e.g. 
visual + auditory) information in task performance. 

4.2.3 The social brain: face perception in childhood 

Human face perception has been the focus of many 
neuroimaging and behavioral investigations in adults, 
in infants, and throughout childhood and adolescence. 
Why is this area of research important to the field of 
cognitive neuroscience? Investigating brain regions 
that may be specialized for perception of our species- 
specific faces may shed light on the nature versus nur¬ 
ture debate. Are we predisposed to attend to, focus on, 
and interpret cues in faces? Or does our vast experience 
with faces provide the information processing abilities 
that are not specific to faces, but rather utilizes visual 
object perception networks? Studying face process¬ 
ing during development may help us determine how 
genetic predisposition interacts with experience. 

Recall that, in infants, investigations of face process¬ 
ing used ERP measures and, in particular, evaluated 
the modulation of the N170 component in response 
to faces as compared to other visual objects. Results 
from these studies as well as studies using other imag¬ 
ing techniques (MEG, fMRI) provide converging evi¬ 
dence that face processing is less specialized in young 
children (5-10 years) as compared to older children 
(11-14 years), with more diffuse and less focal brain 
activity for faces (Gathers et al., 2004; Aylward et al., 
2005; Kyllainen et al., 2006). Also, older children show 
more bilateral fusiform face area activation for faces 
as compared to houses than younger children, and 
the pattern of activation correlates with age (Aylward 










500 


15. DEVELOPMENT 



Executive attention 



0.4 



-0 21 - -- - 

adults children 

Group 


icongruenl targets 
lincongruent targets 



Group 



(C) 


(d) 


FIGURE 15.35 Differential activation of adults and children as identified in a two-sample test for the executive control condition 
(thresholded at p svc < 0.05 or p < 0.1 corrected for multiple comparisons for whole-brain analyses, extend threshold 5 voxel, shown on aver¬ 
aged group T1 image), (a) Increased activation in the superior parietal cortex and (b) inferior frontal gyrus in adults compared to children, 
(c) Increased activation in the superior temporal gyrus and (d) superior frontal gyrus in children compared to adults. Plots of the percent 
BOLD signal change are shown separately for both groups as a function of target type (pooled over cueing conditions) for the activation 
maximum. Source : Konrad et al, 2005. 


et al, 2005). Nevertheless, these studies indicate that, 
while older children show more focal activation than 
younger children, they do not show adult-like pat¬ 
terns of response, indicating that face perception is a 
slow-to-mature process. 

Most of these studies use faces with neutral or 
happy expressions; however, they leave open the 
question of how children perceive emotional infor¬ 
mation in faces. Understanding emotional cues in 
facial expressions is a vital aspect of social cognition 
in humans. The study of face processing of emotional 
cues is complex, however, and not well understood 
in adults let alone in developing children. Here, we 
highlight two recent studies of emotional cues in faces 
with children: one study uses fMRI to elucidate brain 
regions employed in decoding emotional face cues 
and the second utilizes the EEG N170 component to 
investigate the time course of emotional cues in face 
processing. It is important to note that the study of the 
developmental path of emotional cues in face process¬ 
ing is in its early stages: while these studies provide 
some useful early data for understanding how these 


processes develop and mature, much more work is 
needed before we can claim to understand fully the 
intricacies of human face perception. 

Batty and Taylor (2006) investigated the N170 
response for faces showing six different emotions 
(happiness, surprise, sadness, fear, anger, and disgust) 
as well as a neutral face in a large sample (82) of chil¬ 
dren between the ages of 4 and 15 years. Batty and 
Taylor reported that only the oldest group of children 
(14-15 years) showed N170 patterns of response that 
were similar to adults. And, even in this group, the 
N170 response did not appear to be adult-like in dis¬ 
criminating between emotional cues. They suggested 
that interpreting complex emotional information in 
human faces was a slow-to-mature process that was 
not fully realized even in middle teenage years. 

A related study was conducted by Lobaugh and col¬ 
leagues (Lobaugh et al, 2006) using fMRI and the same 
eight emotional categories. In this study, the children 
were 10-12 years old. Lobaugh and colleagues used a 
gender decision task for investigating emotional cues 
in faces: the children were asked to decide if the face 
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FIGURE 15.36 Face versus look results, (a) Regions in blue showed more activity on face trials than look trials. The response was 
reversed for regions in yellow/red. Arrows (b-e) highlight peak clusters. Mean percentage change (±SEM) in peak voxel response across the 
block is shown for the selected clusters, (b) A large expanse of right fusiform/parahippocampal gyrus (20/40/16) was more active in the face 
task, (c and d) Bilateral superior temporal gyrus (c: 48/32/+4, d: +44/32/+8). (e) Posterior temporal gyrus (44/52/4) showed decreased activ¬ 
ity to faces. Source : Lobaugh et al., 2006. 


was male or female. In this way, Lobaugh and col¬ 
leagues attempted to understand the neural substrates 
of implicit recognition of emotional cues during face 
processing. First, Lobaugh and colleagues evalu¬ 
ated brain areas that were differentially responsive to 
faces versus a looking condition. These are shown in 
Figure 15.36. These areas were right fusiform/para¬ 
hippocampal gyrus, which were more active in the 
face task, and bilateral superior temporal gyrus and 
posterior temporal gyrus, which showed decreased 
activity in the face task. 

A second key finding was that brain activation to 
negative emotions (disgust, fear) was stronger than 
responses to positive (happiness) or neutral emotions 
(Figure 15.37). The strongest responses were to faces 
with expressions of fear. The authors interpreted these 
findings to indicate that functional emotional net¬ 
works are established by age 10. 

What do the findings of the work of Batty and 
Taylor and Lobaugh and colleagues tell us about 
how children understand complex emotional cues in 
faces? It is important to note that the brain measures 
used by the two studies (ERP study of the latency 
and amplitude of the N170 component versus fMRI 
study of hemodynamic changes) are quite different 
in just what brain responses they are able to measure. 
Next, one study (Batty and Taylor, 2006) had a large 
sample (82) of children across a wide range of ages 
(4-15 years), while the other had a small sample (10) 


of children in a narrow age range (10-12 years). The 
differing tasks and techniques used in these studies 
make an interpretation of the findings problematic in 
spite of the fact that the two studies investigated very 
similar processes, emotional cues in faces. 

The investigations presented here provide early evi¬ 
dence for differing patterns of brain response in child¬ 
hood when processing emotional cues in faces. Many 
more studies are needed in order to for us to under¬ 
stand fully the brain area and developmental path of 
emotional cue processing in face perception. Some 
important aspects of the future studies may entail 
determining if boys and girls process emotional cues 
in a similar way and assessing if children pay more 
attention to certain aspects of faces (such as eyes) as 
compared to adults. 

What have we learned about the development of 
language, executive function and social cognition in 
childhood and adolescence? One central finding from 
a variety of data sources (histological, pathological, 
neuroimaging) is that the cortical regions subserving 
these high order cognitive functions have a prolonged 
developmental path extending to mid- to late adoles¬ 
cence. We have a wealth of behavioral data showing 
a similar pattern in tasks that tap more complex and 
higher order aspects of these cognitive functions, with 
task performance not reaching adult-like levels until 
late adolescence. Putting these two bodies of informa¬ 
tion together in order to map brain development onto 
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FIGURE 15.37 Emotional faces results, (a) Task contrasts for the first latent variable. The first pattern most 
strongly distinguished disgust (blue, negative task weight) from fear (red, positive task weight) and, to a smaller 
extent, from surprise (cyan) and happiness (gray), (b) Brain regions whose activity differentiated fear and disgust. 
Yellow/red: regions where the response to fear was greater than that to disgust; blue: regions where the response to 
disgust was larger than that to fear, (c-f) Location and mean percentage change (7 SEM) in peak voxels for selected 
clusters (mean of 8 TRs after trial onset), (c and d) Bilateral amygdala (c: +20/4/8; d: 24/4/8). (e) Right superior 
temporal gyrus (+48/40/+8). (f) Left superior temporal gyrus/inferior supramarginal gyrus (48/40/+20). (g) Task 
contrasts for the second latent variable. This pattern most strongly distinguished disgust and fear (blue and red, 
positive task weights) from sadness (green, negative task weight). Surprise (cyan) and happiness (gray) also differed 
from disgust and fear, (h) Brain regions whose activity most strongly differentiated fear and disgust from sadness. 
Yellow/red: regions where the response to disgust and fear was greater than that to sadness; blue: regions where the 
response to sad faces was larger than that to disgust and fear, (i-k) Mean percentage change in peak voxels (7 SEM) 
for selected clusters, (i) Left superior temporal gyrus (44/28/+8). (j and k) Anterior cingulate (j: +12/ +36/ +24, BA 
32; k: +8 / + 8/ + 28, BA 24). Color bars indicate the magnitude of the voxel stability. Left is on the right of the image. 
Source : Lobaugh et al., 2006. 
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cognitive function remains a challenge for the devel¬ 
opmental cognitive neuroscientist. 

While we have made significant headway, as data 
presented in this chapter demonstrate, there are many 
complexities in each of these areas of cognition that 
remain undiscovered. While we have a general notion 
of how language emerges in young children, the cor¬ 
tical mapping of complex language systems is still 
being worked out in adults, with less known about the 
patterns of emergent language. Similarly, the field con¬ 
tinues to discover how PFC subserves aspects of tasks 
tapping attentional, planning, and working memory 
systems. The relation of these networks with ability 
or specific task demands is still unresolved in adults 
and remains to be elucidated in children. Face percep¬ 
tion and the integration of emotional, linguistic, and 
pragmatic cues faces into social cognition knowledge 
is also an ongoing field of investigation in adults, with 
less known about the emergent patterns and trading 
relations among these many factors in understanding 
human social behavior. 

An important direction in the field of developmental 
cognitive neuroscience is to combine methodologies - 
for example, fMRI with EEG - with behavioral measures 
in order to provide converging evidence across meth¬ 
odologies and measures regarding aspects of higher 
order cognitive function. Combining fMRI with EEG, 
for example, can provide high resolution spatial infor¬ 
mation regarding brain activations coupled with high 
resolution of the time course of that activation. Another 
important direction in the field is to conduct longitu¬ 
dinal studies in order to track the development over 
time of individual children. In this way, early 'baseline' 
measures can be taken and then the development and 
change in these measures may be assessed at specific 
intervals. Finally, new experimental design approaches 
with young infants are demonstrating that babies know 
and understand a lot more about the world around 
them than we previously thought. New advances in 
measuring infant cognition and mapping the relevant 
brain activity will provide important insights into the 
developmental changes occurring in the first year of life. 


5.0 EARLY BRAIN DAMAGE AND 
DEVELOPMENTAL PLASTICITY 

We mentioned, above, the importance of longitudi¬ 
nal studies for tracking individual progress and out¬ 
comes throughout development. This type of study 
is especially important when assessing the long-term 


outcome of early (perinatal) brain injury. We have 
seen in earlier chapters that, in adults, brain damage 
due to stroke, disease, or traumatic accident typically 
leads to deficits in aspects of cognition that are fairly 
severe, with complete recovery of function unlikely. 
What happens when brain damage occurs near birth? 
This question has an important bearing on the nature 
versus nurture debate. Consider the hypothesis that 
some brain systems, such as language, have a strong 
genetic predisposition for their development in speci¬ 
fied regions of the brain. If there is early insult to those 
pre-specified regions, will the infant develop language 
in a typical fashion? Or will language develop in an 
aberrant fashion due to the early and unrecoverable 
damage to those brain regions? Alternatively, if expe¬ 
rience plays the dominant role in the development of 
brain regions that become tuned for language func¬ 
tion, will the infant develop language in a typical fash¬ 
ion in spite of the early brain damage? 

The effects of perinatal brain damage have been 
extensively investigated in animal studies in the field 
of neurobiology. The effects of early brain damage 
and the impact on later cognitive development have 
been far less studied in humans. One reason for this is 
that a single, unilateral (in one hemisphere only) pre- 
or perinatal brain insult is relatively rare. Typically, 
instances of early brain insult are more global in 
nature and combine with other neurological complica¬ 
tions (Figure 15.38, left panel). In these cases of larger 
scale damage coupled with other traumatic events, it 
is difficult to compare cognitive development to chil¬ 
dren without this early damage and trauma. In cases 
where the perinatal damage is limited to a circum¬ 
scribed region (Figure 15.38, center and right panels), 
the long-term effects are typically milder. These are 
the types of cases that we will focus on in this section: 
early, focal, unilateral brain insult. 



FIGURE 15.38 Large scale and smaller scale perinatal brain 
damage. Structural MRI scans in the axial plane from three children 
with perinatal brain damage, illustrating different patterns of injury. 
Left: A large unilateral lesion involving most of one cerebral hemi¬ 
sphere. Middle: A small lesion confined to one cerebral lobe. Right: A 
deep lesion involving subcortical regions. Source: Stiles et al., 2005. 
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While we are just beginning to understand the com¬ 
plexities of early insult on later cognitive growth in 
humans, a series of longitudinal studies by Stiles and 
colleagues (2005) shed some light on the long-term 
effects of perinatal insult and we highlight some results 
here (for a review, see Stiles et al., 2005). The San Diego 
Longitudinal Project (Stiles et al., 2002) is the largest 
U.S. investigation of the long-term effects of perinatal 
brain damage. Stiles and colleagues have followed the 
cognitive development of several hundred children 
with perinatal brain damage since 1989. 

Much of the focus of the investigations by Stiles 
and colleagues in the San Diego Longitudinal Project 
has been on language development in children with 
perinatal brain damage. One key finding is that, while 
in adults, focal brain damage (typically through lesions 
due to stroke) in language centers results in long-last¬ 
ing deficits, this pattern is quite different with infants 
who suffer perinatal brain damage. While there is typi¬ 
cally delay in early linguistic milestones, such as onset 
of word comprehension at 9-12 months and word pro¬ 
duction at 12-15 months, by the age of 5 years, these 
children have largely 'caught up' in linguistic abilities. 
The important finding, however, is that when tested 
carefully there remain some underlying deficits even 
at the age of 5, especially in complex sentence struc¬ 
tures (Reilly et al., 2004). These children with early brain 
damage do ultimately achieve language competence, 
but the evidence provided by the longitudinal studies 
by Stiles and colleagues (reviewed in Stiles et al., 2005) 
indicate that their language proficiency is in the lower 
than normal range. Thus, while the children do acquire 
many skills and proficiencies with respect to language, 
there remain throughout childhood, adolescence, and 
presumably adulthood, some key deficits due to the 
very early damage to important brain regions for lan¬ 
guage acquisition and processing. 

Another aspect of cognition that has been the 
focus of study by the San Diego Longitudinal Project 
has been spatial cognition. Spatial cognition and the 
effects produced by adult acquired brain damage have 
been the target of many neuropsychological and neu¬ 
roimaging investigations. The central findings have 
been that there is a hemisphere asymmetry in the 
decoding of visual patterns, with the left hemisphere 
biased for extracting feature (local) information and 
the right hemisphere biased for extracting configura¬ 
tion (global) information (Figure 15.39). 

Functional MRI studies of typically developing 
adolescents show that they demonstrate a similar 
hemisphere asymmetry, with greater right hemisphere 
occipital-temporal activation for global processing and 
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FIGURE 15.39 Global versus local: examples of visuospatial 
deficits. Examples of memory reproductions of hierarchical form 
stimuli by adult stroke patients with either right or left hemisphere 
injury (Adapted, with permission, from Delis et al., 1986.) The sam¬ 
ple stimulus to be copied is shown on the left side of the figure. 
Center of the figure: patients with right hemisphere damage typi¬ 
cally produce the local (detailed) aspects of the stimulus but omit 
the global (overall) aspects of the stimulus: in this case, the 'M' or 
triangle shape of the stimulus. Right side of the figure: patients with 
left hemisphere damage typically produce the global aspects of the 
stimulus but omit the local aspects of the stimulus. Source : Stiles 
et al., 2005. 


greater left hemisphere occipital-temporal activation 
for local processing (Figure 15.40). In stark contrast, a 
15-year-old adolescent who had suffered right hemi¬ 
sphere perinatal brain damage showed stronger acti¬ 
vation for both global and local processing in the left 
(undamaged) hemisphere (Figure 15.40) and a 13-year- 
old adolescent who had suffered left hemisphere peri¬ 
natal brain damage showed stronger activation for both 
global and local processing in the right (undamaged) 
hemisphere (Figure 15.40). Thus, the fMRI data for the 
two adolescents who suffered perinatal brain damage 
provide evidence for long-lasting damage to spatial 
cognition mechanisms. However, they also provide 
intriguing evidence for a brain system that is highly 
flexible, with recruitment of neural territory in the 
undamaged hemisphere for spatial cognition functions. 

What have we learned about the long-term effects 
of early brain damage in the longitudinal studies 
of Stiles and colleagues? And how do they inform 
us about the complex and highly interactive roles of 
nature and nurture in human development? While 
these efforts are still in the early stages, results to date 
indicate that early brain damage results in long-term, 
though typically somewhat subtle, deficits. A second 
important finding is that despite the early insults and 
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FIGURE 15.40 Global and local processing in the brain. Functional MRI activation data from two teenagers with prenatal focal brain 
injury on a hierarchical form-processing task, compared with data from typical adolescents. Each child participated in separate imaging runs, 
where they were asked to attend to either the global or the local level of the stimulus pattern. Unlike typical controls, who show different pat¬ 
terns of lateralization for global and local processing, the two children with lesions showed activation largely confined to the uninjured hemi¬ 
sphere. Activation images for the two children with perinatal brain injury are shown. Source : Adapted from Stiles et al., 2005, with permission. 


the delays that they typically produce in cognitive 
development, the children mature and acquire higher 
cognitive function, although sometimes at the lower 
than normal level. Cumulatively, these findings pro¬ 
vide evidence that there are some brain systems that 
have long-term impairments when damaged, even 
when the damage occurs at or near birth. This pro¬ 
vides some support that some systems have a level of 
genetic predisposition and can suffer long-term harm 
when disrupted. On a brighter note, these findings 


provide evidence for significant amounts of early 
brain plasticity so that the cognitive functions that suf¬ 
fer early damage develop in an alternative manner. 

6.0 SUMMARY 

In this chapter, we have tracked the stages of human 
development from early embryo to infant to ado¬ 
lescent. While the field of developmental cognitive 
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neuroscience is still a very young one, nevertheless, 
the findings presented in this chapter demonstrate the 
answers to important questions about human brain 
development and the correspondence to cognition. An 
overarching topic of much debate in the field of human 
development is the role of nature versus nurture. From 
the data presented here, you see that at each stage of 
human development there are important genetic effects 
and biological constraints at work in the unfolding of 
the human brain and mind. Similarly, at each state there 
are critical effects of the surrounding environment, 
whether at the level of the cell, the system, or the brain. 


The advent of new techniques for non-invasively 
studying human development has provided the means 
to address new questions about cognitive develop¬ 
ment, such as what does a baby know before birth? 
Does an infant understand the grammar of language? 
How does the sense of self develop in an infant, a 
child? What are the long-term effects of focal brain 
damage? These and other questions will be addressed 
in future studies investigating the unfolding complex 
pattern of human brain development and its relation 
to cognition. 


7*0 CHAPTER REVIEW 
7*1 Study questions 

1 In what ways have neuroimaging techniques 
changed the way infant and child development 
is investigated? 

2 What does the term 'bidirectional influences' 
refer to in human development? Why is it an 
important concept? 

3 Provide an example of a nature and nurture 
interaction that occurs before birth (prenatally). 


4 What are some effects of maternal use of alco¬ 
hol, tobacco, and marijuana on her unborn 
child? 

5 What brain regions develop and mature early 
in childhood? What regions develop later in 
childhood? 

6 The development of object permanence in infants 
has been the focus of much study. Describe an 
experiment for investigating object permanence 
and possible results for: 

a a 6-month-old infant, 
b a 12-month-old infant. 
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The human brain has been creative ever since the emergence of behaviorally modern people. Lascaux cave painting, estimated to be 16 000 
years old. Source : http://en.wikipedia.org/wiki/Lascaux 
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16. THE GENES AND MOLECULES OF COGNITION 


1.0 INTRODUCTION 

What do chocolate candy and the sting of a bee have 
in common? Both act by way of signaling molecules 
between brain cells. Chocolate tastes good and insect 
bites itch - both are brain responses. A great variety of 
everyday molecules shape our experiences, including 
coffee, aspirin, vitamins, beer, oils and fats, medicines, 
sugar, nicotine, and diesel fumes. Insect bites and 
snake poisons involve neurotoxins , chemicals that inter¬ 
fere with neural transmission. Eggs provide choline, 
which is used to make the neurotransmitter acetylcho¬ 
line, a molecule needed to stay awake. Milk contains 
tryptophan, the first step for producing serotonin, which 
is why a glass of warm milk at night may promote 
sleep. Coffee, tea, and chocolate all have some caffeine, 
which inhibits the neurotransmitter adenosine by com¬ 
peting for its receptors (Figure 16.1). Neurotransmitters 
are ever-present players in everyday life. 

Ordinary food and drink is full of neuroactive mol¬ 
ecules, including sugars (glucose being the food of the 
brain), choline (from eggs), and whole proteins (from 
milk, meat, eggs, nuts, and seeds). Calcium is utilized 
for a huge variety of cell functions, including neural 
transmission. Ultimately, we get all those building 
blocks from the food we eat and the air we breathe. 

All living things are made of carbon chains, which 
act as Lego bricks that can make up an endless vari¬ 
ety of buildings and figures. The carbon chains utilize 
hydrogen, nitrogen, oxygen, calcium, magnesium, 
and trace minerals. Proteins are long chains of amino 
acids, carbon-hydrogen chains using the NH 2 and 
COOH fractions. All amino acids can be made from 
whole protein foods, which provide the twelve 'essen¬ 
tial' amino acids. 



Caffeine Adenosine Theophylline 

FIGURE 16.1 Coffee, tea, or adenosine? The three-dimensional 
shape of molecules determines many of their effects. Caffeine and 
theophylline both resemble the neurotransmitter adenosine in their 
chemical shapes, and by blocking the adenosine receptor they 
increase alertness. Adenosine is also part of the basic biological 
energy molecule ATP (adenosine triphosphate); neurotransmitters 
often make use of existing biomolecules. 


1.1 Basic life molecules are also used for 
neuronal signaling 

Scientists often seem to talk in their own code, and the 
vocabulary of brain molecules can seem overwhelm¬ 
ing. To remember the words it helps to look for roots. 
The word cannabinoid (ka-NA-bin-oid) comes from 
the Latin name for the psychoactive molecules of the 
hemp plant. Cannabis sativa, with the ending-ozd to 
express the fact that those molecules are Tike' the can¬ 
nabis we find in plants. Endo-cannabinoids are endog¬ 
enous cannabis-like molecules, produced inside the 
body itself. 

Brain molecules often are named after their first 
known role in the body. Thus serotonin was first 
described as a substance that raised blood pressure 
(the tonus of the serum; pronounced SE-ro-TOE-nin). 
The ending '-in' or '-ine' indicates a protein, as in the 
word 'protein' itself. Though serotonin does raise 
blood pressure when it enters the bloodstream, in the 
brain it helps to run the sleep-waking cycle, along 
with regulating mood and dreams. But it still keeps its 
original name, giving us a useful trick to help remem¬ 
ber it. Like street signs, chemical names commonly 
carry their own history. 

Memory techniques can be very efficient. To 
remember the words you can look for similar ones that 
you already know, even if the association is imaginary. 
(If 'serotonin' sounds like 'cherry toenails' to you, that 
will work fine, especially if you can visualize it.) For 
spatial facts like the landscape of the brain and the 
layout of synapses, nothing works as well as draw¬ 
ing and coloring. Learning is an active and interactive 
process, and the more you can actually do with the 
terms in this chapter, the better you will remember 
them. If you can actively reconstruct the main points in 
your own words, you will have a solid foundation of 
understanding. 

The brain itself supplies some simplifying rules of 
thumb. One is that life constantly uses the same mol¬ 
ecules in different ways. Most of the molecules we dis¬ 
cuss in this chapter exist across many animal species, 
from the fruit fly and the snail-like Aplysia to human 
beings. Molecules that are involved in energy metabo¬ 
lism and heredity, basic life functions, are constantly 
reutilized for neuronal signaling. All animal cells 
maintain a voltage difference across their cell mem¬ 
branes, but only neurons use that voltage to send an 
action potential (spike) down the axon. All cells use 
ATP to store energy, and ATP is also reutilized as a 
neurotransmitter in its own right. Glutamate is a basic 
amino acid in all cells, but it is also the most common 
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excitatory neurotransmitter in the brain. There are 
numerous other examples. The fact that nature reuses 
its basic molecules to run nervous systems helps in 
learning the vocabulary. 

Simpler animals are intensively studied as model 
organisms to work out the most basic biological events. 
Synapses are tiny and complex, and historically required 
invasive surgery to study, not something that is ethically 
permitted in humans. They were therefore first studied 
in simpler animal models and even in laboratory cul¬ 
tures. Those tools continue to reveal new facts. 

Animal biology shows a number of highly con¬ 
served genes, molecules, and nerve cells. By conserved, 
we mean that they are found across many species. For 
example, we share five major classes of neurotransmit¬ 
ters with insects. That does not reduce human beings 
to fruit flies, but it makes the fruit fly useful as a 
model to study. The electrical brain activity of a mov¬ 
ing fruit fly actually resembles the EEG of the waking 
brain in humans. Such similarities across hundreds of 
millions of years of divergent evolution are part of the 
fun of learning the biology of brainy creatures. 

Looking closer to home, other mammals share ear¬ 
lier evolutionary structures with us. The neocortex, 
and especially the frontal lobes, have grown on those 
shared foundations. The brain contains its own evolu¬ 
tionary history. 

Molecules work at the most powerful level of bio¬ 
logical control. Genes are encoded in molecules, the 
nearly two meters of DNA in each cell, which is tran¬ 
scribed into messenger RNAs, which are then translated 
into target proteins (Figure 16.2). 

The brain cannot work without a constant flow of 
oxygen and glucose, about a quarter of the body's 
energy supply. It also needs regular protein, vitamins, 
minerals, oils, and fats - the last are required to build 
cell membranes and the myelin sheath surrounding 
neurons. Some oils have been shown to raise mood 
(the omega 3 oils). Molecules like dopamine and ace¬ 
tylcholine are essential for pleasure, for staying awake, 
and for immediate memory. 

The molecular machinery of the brain can seem over¬ 
whelming. This chapter will cover the basics. In spite 
of the dizzying dance of molecules we can still think of 
all the steps as either increasing or decreasing the func¬ 
tion of a specific target chemical. A dopamine agonist 
is anything that increases the functions of dopamine. 
Thus the famous Parkinson's drug, L-dopa, is an ingre¬ 
dient for making dopamine, and therefore can act as 
a dopamine agonist. SSRI antidepressants - selective 
serotonin reuptake inhibitors - reduce the reuptake (recy¬ 
cling) of serotonin. If you don't recycle a molecule you 


get more of it. So the overall effect of SSRIs is also to 
be an agonist of serotonin. Simply counting the ups 
and downs of each step will give you the overall story. 
It may be helpful to write down small up and down 
arrows for each step (| and |) to keep track of the 
whole series of events. 

On the other side of agonists, antagonists lower the 
effects of a neural messenger. A prominent example is 
the drug naloxone, which is used to reverse the deadly 
effects of an opiate overdose. Naloxone competes for 
opiate receptor slots, and thereby reduces the effects 
of toxic quantities of opiates. Like other neural mol¬ 
ecules, opiates have important uses as well as risks. 

In sum, no matter how complicated the molecu¬ 
lar dance becomes, a psychoactive chemical ends up 
either increasing or decreasing the effects of some 
brain molecule. 

This chapter explores the molecules of cognition. 
We want to know how molecules help to carry brain 
signals and perform the computations that make the 
human mind possible. 

1.2 Life uses enzymes 

Living organisms operate within very narrow limits of 
temperature, pressure, and acidity. Humans learned to 
cook their food about 10 000 years ago, and to make 
high-temperature steel about two centuries ago, in 
order to make most of our technology. But the body 
cannot tolerate extremes of heat, pressure, or acidity; 
that would kill us. Instead, life uses enzyme chemistry. 



FIGURE 16.2 DNA and the enzymatic chemistry of life. In 
a broad sense all biochemical players are controlled by enzymes, 
including DNA itself. Enzymes enable chemical reactions by locking 
into substrate molecules, like a key in a lock. They can cleave, trans¬ 
port, and release other molecules, very much like nanomachines. 
Enzymatic reactions are more efficient than simple chemical reactions, 
and have major amplifying effects. DNA is both a code-preserver and 
a source of endless molecular amplification. Source: National Institute 
of General Medical Sciences, 2009. 
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making constant use of molecular nanomachines that 
speed along a second-by-second flow of chemical reac¬ 
tions. All brain chemicals act enzymatically. 

The easiest way to think about enzymes is with 
the traditional 'lock and key' analogy; it is the three- 
dimensional shape of the molecules that allows them 
to lock into each other, transport molecules, change 
their shapes, or use them to send signals. Vitamins 
are essential enzymes to sustain life (from Latin, vita). 
Vitamin B12 and folic acid have important brain func¬ 
tions. Neurotransmission consists of enzyme-mediated 
signaling. 

1.3 The same molecule may send 
different signals 

The hemp plant does not produce cannabis to pro¬ 
vide interesting experiences for humans. The coffee 
plant did not evolve to make coffee for your breakfast. 
Those molecules play different roles in their original 
plants than those they trigger in the brain. 

But why are plant molecules so powerful for 
humans? Many useful drugs were first isolated from 
plants. Aspirin was derived from the bark of the wil¬ 
low tree. Curare comes from the arrow poison used 
by native people in Guyana, and played a major role 
in the discovery of one of the first known neurotrans¬ 
mitters, acetylcholine. Thousands of neuroactive mol¬ 
ecules originally were derived from plants or other 
animals. The chemistry of life is highly conserved : the 
same molecule may act as an insecticide for a plant 
and as a neurochemical in animal brains. 

A molecule may do different things even in the 
same animal. Adrenalin stimulates the body to prepare 
for Tight or flight/ raising heart rate and breathing, 
and channeling blood to the lungs and outer mus¬ 
cles. Adrenalin is secreted from a gland on top of the 
kidneys (from Latin, ad = on top of, renes = kidney, 
-in = protein). In the brain it is called epinephrine 
using the same linguistic roots (Greek epi = on top of; 
nephros = kidney; -ine = protein). 

In the brain norepinephrine is a major neuromod¬ 
ulator for wakefulness, different from the hormonal 
functions of adrenalin in the body. Similar molecules 
play different roles in different parts of the body. 

1.4 The blood-brain barrier protects the 
internal milieu of the brain 

One way for adrenalin to play one role in the body and 
another one in the brain is by compartmentalizing the 


distribution of the molecule, to ensure that when you 
get excited about a football game your brain doesn't 
overload with epinephrine. The blood-brain barrier was 
discovered shortly before 1900 when a chemist noticed 
that a blue dye injected in the brain did not spread to 
the body, or vice versa. Figure 16.3 shows some major 
features of the blood-brain barrier, which lines the 
blood vessels and ventricles of the brain. 

The blood-brain barrier is a wall of separation 
between the body and its brain. It allows the brain 
to control its own internal milieu, separate from the 
bloodstream. The body itself is protected by layers 
of skin and muscle to separate it from surrounding 
chemicals. In the upshot, every cell in the brain is sur¬ 
rounded by many layers of barriers - the cell mem¬ 
brane itself, surrounding glial cells, the blood-brain 
barrier, the tissue barriers surrounding the brain, and 
the skin barriers to the outside world. Inside a brain 
cell, the nucleus constitutes yet another encapsulated 
organelle, and the nuclear chromosomes provide a 
protective medium to preserve the integrity of DNA. 
It's all biocontainers within biocontainers. 

The brain and spinal cord have a private circula¬ 
tory system separated from the bloodstream, called 
the cerebrospinal fluid (CSF). CSF serves to transport 
molecules and cells. A constant but controlled flow of 
chemical traffic flows to the brain by way of the CSF 
and the bloodstream. 

Hormones are defined as signaling molecules car¬ 
ried to bodily organs via the bloodstream, while neu¬ 
ral messengers lock on to receptors in the brain. But 
the brain is also richly vascularized with blood vessels, 
large and small. As we have seen, the fMRI method of 
brain recording picks up blood-oxygen-level-dependent 
(BOLD) changes in the magnetic signature of a brain 
region, when more blood is directed there to sup¬ 
ply neurons and glial cells to run a specific task. The 
blood-brain barrier is a selective barrier, allowing oxy¬ 
gen and glucose to diffuse from the bloodstream, but 
blocking larger molecules like adrenaline. 


T5 Model organisms 

Peas, insects, worms, and rats hold a special place of 
honor in science because they have long served as test¬ 
beds for studies that cannot be done in people (Figure 
16.4). Each model organism rose to fame for biological 
discoveries that were difficult to observe elsewhere. 
Today the round worm C. elegans has come to the fore¬ 
front, in part because we can identify every neuron 
in its body. Biological mechanisms are often highly 
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(a) Neurovascular unit 
(blood-brain barrier) 


(c) Meninges 

(arachnoid barrier) 



FIGURE 16.3 The brain is wrapped in protective layers of cells. The blood-brain barrier lines blood vessels in the brain as well as 
the cerebrospinal fluid vessels. The brain therefore has its own protected internal milieu, separate from the bloodstream, which in its 
turn is separate from other parts of the body and the environment. Large molecules, like proteins, are blocked unless they are specifi¬ 
cally recognized and transported across the barrier. Similar cellular walls shield the fetal brain and the adult cranium from unwanted 
molecules. Source : Saunders et al., 2008. 


conserved, so that we can learn about the human brain 
from the things it has in common with fruit flies and 
mice. Synaptic learning was first studied in the giant 
axon of the sea hare Aplysia because it was big enough 
to study under the light microscope. It is only today, 
decades later, that we are seeing direct evidence for 
synaptic learning in the human brain. In a sense, in 
biology we always start by looking in the wrong place. 


It is therefore vital to keep an open mind, to expect the 
unexpected. 

The scientific study of peas and fruit flies began 
before 1900. Since the biogenetic revolution of the past 
few decades, numerous experiments have utilized 
'gene knockout' and 'knockin' organisms. Using retro¬ 
viruses it is possible to insert new genes into organ¬ 
isms, and to delete selected stretches of DNA. While 
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the mass of accumulated evidence is historic, there is 
still no substitute for clear thinking. Neurobiology is 
not a settled science - surprises keep happening. 

Historically, acetylcholine (a-SEE-til-KO-leen, abbre¬ 
viated ACh) was the first neurotransmitter studied. 
ACh was discovered in the neuromuscular synapse of 
experimental animals, outside of the central nervous 
system. The discovery of ACh had immense influence. 
For example, it was found to have two different recep¬ 
tors with quite different effects, a finding that alerted 
researchers to look for multiple receptors for other mol¬ 
ecules. One of the ACh receptors was called nicotinic 
because it responded to nicotine, derived from tobacco. 
The second receptor was called muscarinic , because 
it was activated by the toxic ingredient of the poison 
mushroom, Muscarina amanita. 

ACh itself has two different guises: outside of the 
central nervous system it is a neuromuscular trans¬ 
mitter, but in the brain it is a major neuromodulator, 
with effects that are quite different from its peripheral 
role. Our understanding of acetylcholine therefore has 
changed fundamentally. In the brain, a cholinergic 
(ACh) system triggers the brain activity for waking 
cognition and REM dreams. Again, the same molecule 
has two radically different roles. 



FIGURE 16.4 The mouse cortex: measuring gene expression. 
Whole-brain gene expression in the mouse. In this figure, a mouse 
brain is injected with a fluorescent reporter dye that reflects neuro¬ 
nal gene expression by the ARC gene promoter, which is involved 
in learning and neuroplasticity. Discoveries in other species often 
cast new light on the genetic control machinery of the human brain. 
Source : Eguchi and Yamaguchi, 2009. 


The ACh synapse also revealed an enzyme to deac¬ 
tivate ACh. Acetylcholinesterase degrades ACh after it is 
released. Clean-up is a crucial part of synaptic chem¬ 
istry, another important point. Without clean-up the 
synapse would be poisoned by excess ACh. Bioactive 
molecules are tightly regulated in the body. (The end¬ 
ing -ase stands for an enzyme that inactivates a mol¬ 
ecule, in this case ACh.) 


2.0 GENES IN EVOLUTION, THE 
LIFESPAN, AND DAILY EVENTS 

Genetics was controversial for many years because 
speculative ideas about racial and ethnic inheritance 
have been badly abused. It is important to approach 
it with great care and precision, and with considera¬ 
tion for all of us, because we are all influenced by our 
genes. Human differences now are viewed mainly 
as signs of the continent of origin of our more recent 
ancestors. But these visible differences came about 
fairly recently. Our hominid ancestors evolved in 
Africa over millions of years (Figure 16.5). It was only 
30 to 70 thousand years ago that Homo sapiens sapiens 
began to spread from Africa to the rest of the world; 
migrations inside Africa were also very common. The 
modern human genome can be dated back to a popula¬ 
tion 'bottleneck' of more than 100,000 years ago, when 
the proto-human population was believed to have 
been reduced to about 5000 individuals. Such bottle¬ 
necks tend to restrict genetic variation, so that today 
individual human genomes differ only by 1 or 2%. 

Many psychological and brain variables are roughly 
50% heritable. The question today is which specific 
genes do what - with regard to disease risk, for exam¬ 
ple. Humans have biochemical individuality, as reflected 
in our different responses to life-saving medications. We 
are just beginning to understand many of those issues. 

Our Homo sapiens ancestors were almost certainly 
dark-skinned, as an adaptation to the intense sunlight 
of the equatorial regions. Sunlight destroys folic acid in 
the skin, a crucial B vitamin for reproduction. Excessive 
sunlight can also cause skin cancer, though a moder¬ 
ate level of sunshine is needed to make the essential 
vitamin D. In sunnier regions of the world humans 
evolved higher levels of the skin pigment melanin to 
provide protection from sunlight. It is melanin that tans 
the skin, and it is differences in melanin that account 
for different skin colors among human populations. 

There is speculation that the first 'behaviorally mod¬ 
ern humans' - defined as people who used symbolism. 
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tools, and fire and had a spoken language - may have 
resembled the San people of South Africa. However, 
human brains are strikingly similar across the con¬ 
tinents. Our external differences reflect the adaptive 
histories of human ancestors in different conditions, 
coping with different temperature extremes, disease 
vectors, social conditions, and nutrients. Dairy foods 
require a metabolic adaptation to digest lactose (milk 
sugar), and many people are lactose intolerant. Some 
human groups evolved enzymes to metabolize the 
mother's milk of a different species, like cows or goats. 


7 mya 6 5 4 3 2 1 0 



FIGURE 16.5 Four million years of hominid evolution. 
Human-like creatures have dwelled on earth for about 4 million 
years, beginning even before the hominid ancestor named 'Lucy' 
(see above). This diagram shows our current understanding of the 
emergence of Homo sapiens sapiens from ancestral species. Today 
chimps and bonobos are our closest primate relatives (lower right 
corner). But a few hundred thousand years ago, different hominids 
coexisted at the same time (upper-right corner). Source : Kaas and 
Preuss, in Squire et al., 2003. 


Other recent changes emerged to cope with colder tem¬ 
peratures, toxins, and infectious diseases. 

Yet we can see amazing similarities with our ances¬ 
tors from tens of thousands of years ago. The top of 
this chapter shows a fragment of the cave art found 
near the village of Lascaux in southern France around 
1940. It is dated about 16 000 years ago. When Pablo 
Picasso first visited the caves, soon after their discov¬ 
ery, he reportedly said, 'We have learned nothing new' in 
making art. Although we do not know how those early 
people thought, their art gives us a sense of kinship. 

2.1 DNA operates over multiple time scales 

DNA operates over a remarkable range of time scales. 
Genes work over geological stretches of time - hundreds 
of millions of years - as well as generations. For 
humans, a generation is about 15 to 30 years. But 
genes operate over the 24-hour cycle as well. They are 
now known to control the sleep-waking cycle, learn¬ 
ing and neural plasticity, the immune response, aging, 
disease progression, and healing. Your brain is doing 
some gene-based processing right now to enable you 
to learn what you are reading at this moment. 

Keep in mind that genes play three major roles: they 
have shaped the evolution of the brain, such as the 
frontal lobes - including language, social relationships, 
and the ability to recognize facial expressions. They 
control the program for lifelong brain development, 
from fetal growth to aging. And in nerve cells, genes 
also control moment-to-moment events like learning 
and falling asleep. 

Folklore around the world has long dwelled on fam¬ 
ily resemblances. There's nothing new in that. But the 
role of genes in human behavior has been controversial 
in science. Much of that controversy was resolved a few 
decades ago when scientists began to do careful studies 
of twins, siblings, and extended families. Researchers 


TABLE 16.1 Twin studies of neurological disease risk 


Disease 

Monozygotic (MZ) twin concordance 

Dizygotic (DZ) twin concordance 

X s 

Multiple sclerosis 

25% [54] 

5% [54] 

20 

Autism 

75% [60] 

16% [60] 

45 [61] 

Migraine with aura 

50% [62] 

21% [62] 

4 [63] 

Late-onset Alzheimer's disease 

59% [64] 

32% [64] 

5 

Schizophrenia 

50% [65] 

4% [65] 

10 [66] 

Poliomyelitis 

36% 

6% 


Rickets 

91% 

23% 


Tuberculosis 

32% 

14% 



Twin studies helped to establish heritability patterns for genes. For monozygotic (single egg) genes, concordance rates can be quite high and 
somewhat lower for heterozygotic twins. 
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found striking correlations depending upon the degree 
of family relatedness. Identical twins show the highest 
correlations, followed by nonidentical twins, siblings, 
and more distant relatives (Table 16.1). In exceptional 
families certain diseases have been found at much 
higher rates than the average. They are commonly 
studied to explore the molecular basis of disease. 

A famous example is the KE family in the United 
Kingdom. This family demonstrated severe speech 
and language deficits over three generations, called 
verbal dyspraxia , though only about half the family 
members were impaired. Over three generations the 
KE family had a high probability of missing a stretch 
of DNA that came to be called the FOXP2 gene, follow¬ 
ing an inheritance pattern called autosomal dominant. 
A genome scan of family members showed a break in 
chromosome 7 due to a local mutation. 

Speculation became rife that the gene for language 
had been discovered. That would be important. 
Previous chapters have discussed Broca's area (the 
lateral inferior frontal gyrus, usually in the left hemi¬ 
sphere), which is needed for speech production. fMRI 
scans of the KE family showed underactivation of 
broca's area while speaking, but also deficits in 
the basal ganglia and its control of the vocal tract. 
However, the FOXP2 mutation impaired speech com¬ 
prehension as well as production. 

The fact that humans learn language very early in 
life, and that our closest living cousins, the chimps 
and bonobos, do not, suggests there is something dif¬ 
ferent about humans as speakers. Full-blown language 
and symbol manipulation emerged fairly recently 
in human evolution, perhaps as late as 50 to 100,000 
years ago. FOXP2 therefore aroused a great deal of 
interest as a candidate gene for language. 

Some of that excitement faded when it was found 
that the same gene is involved in vocal production in 
songbirds like canaries, as well as bat echolocation, and 
vocalization in mice, monkeys, and even reptiles. Some 
birds are talented speakers, like the African gray parrot, 
but they are not likely to have evolved speech and lan¬ 
guage for the same functions as Homo sapiens. Parrots are 
excellent vocal mimics and intelligent, environmentally 
aware, social creatures. FOXP2 may therefore be involved 
in something humans have in common with parrots - and 
alligators. Its most obvious role might be in the develop¬ 
ment and control of the vocal tract. It now seems that the 
FOXP2 gene is involved in regulating other genes that 
guide the growth of the brain, lungs, and gut. 

It seems that language is not a single capacity. It 
is a large set of capacities, some of which we share 
with other species - like the development of lungs 


and breathing muscles, moment-to-moment control 
of breathing and vocalization, the ability to know the 
world, the ability to act on that knowledge, and to com¬ 
municate it to others. Scientists are now exploring gene 
candidates that may play a role in speech and language. 
That may help to discover molecules to aid people with 
language deficits, and perhaps even to help the rest of 
us remember new vocabularies in brain science. 


2*2 Brain development: neurogenesis, then 
synaptogenesis 

Genes and molecules play different roles in different 
stages of development. It is useful to understand this 
general fact in terms of Gerald Edelman's first and sec¬ 
ond repertoire of neuronal variation and selection. In the 
first repertoire, tens of billions of neurons sprout as the 
fetus grows, each new cell following its own pathway 
to make up the brain, spinal cord, and peripheral neu¬ 
rons. Different classes of cells emerge at this time, until, 
when a baby is born, it will have more nerve cells than 
it will ever again have in its entire lifetime (Figure 16.6). 
That is because neurons are constantly being pruned , if 
they make unstable or unproductive connections. The 
brains of newborns do a lot of pruning, to refine and 
differentiate their connections. This 'amplification and 
pruning' are Darwinian in nature, in that the most 
working neurons and circuits survive, while others die 
off. Thus functional brain circuits begin to emerge early 
in gestation, and by the time a baby is nearing birth, 
most of its hours are spent in a REM-like state, show¬ 
ing an active EEG with rapid eye movements. 

Once a fetus begins to use its senses and learn, around 
the beginning of the third trimester of pregnancy, a sec¬ 
ond repertoire arises. While the flow of new neurons 
slows down, new synaptic connections keep sprouting 
in the trillions. Synaptogenesis now occurs in great pro¬ 
fusion, followed by synaptic pruning. In this second 
stage, Darwinian selection therefore applies to synaptic 
connections rather than nerve cells. The Hebbian rule 
applies in that ' neurons that fire together, wire together' 
(Chapters 3 and 10). Synapses that make useful con¬ 
nections tend to survive, while unconnected ones may 
wither. The connectivity pattern of the brain is the key 
to its working, and this second repertoire of synaptic 
growth and selective pruning continues throughout life. 

In the first repertoire of neurogenesis, when millions 
of neurons are born and develop, genes and molecules 
are building the brain and guiding neuronal branches 
to their targets. Once the basic structure of the brain 
is settled, the same genes and molecules switch to run 
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FIGURE 16.6 Environment and genes over the lifetime. Environment and genes interact in all stages, (a) 
Proliferation: soon after conception neurons multiply and spread, in a process called neurogenesis, (b) 
Differentiation and migration: cells differentiate in shape and function, migrate, and extend long branches to 
reach their final form, (c) Axonal guidance is crucial in this still rather mysterious process, (d) Axons and den¬ 
drites form synapses with other neurons; successful synapses are kept, and unsuccessful ones are pruned, (e) 
Working networks enable information processing in the growing brain. Brain development begins with zero 
synapses and ends with a thousand trillion connections. Source: Yehezkel, 2008. 


the brain on a day-to-day basis. For example, the nor¬ 
mally inhibitory neurotransmitter GABA is actually 
excitatory in early brain development, when it plays 
an important role in guiding axonal pathfinding. 

The result is a maximum of adaptive variation 
(Figure 16.7). Keep in mind that humans, like other spe¬ 
cies, have had to cope with a constant threat of starva¬ 
tion, drought, new diseases, new social challenges, new 
mutations, new varieties of sexual selection - one of 
the basic sources of Darwinian adaptation - and many 
other challenges and opportunities. When humans 
began to herd goats and cows for their milk, people 
who lacked the enzymes to digest goat's milk encoun¬ 
tered a basic challenge to their health. It all depended 
upon their biochemical readiness. 

Genes and molecules bring disease as well as 
health. We will discuss cognitive, social, and neuro- 
degenerative brain disorders. But most genes work 
silently from moment to moment to grow and main¬ 
tain our brains and bodies. 


2.3 How did you get that big brain? 

The human brain expanded rapidly over the past few 
hundred thousand years, much faster than our bodies. 
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FIGURE 16.7 Neurons are born throughout life. A rat's hip¬ 
pocampal view of landmarks in a maze: if it's orange, do I turn left 
for food? The hippocampus has place cells in the CA 1 and CA 3 
regions, which learn to respond to different landmarks, and even to 
monitor the speed and bodily orientation of the animal at various 
choice points in running the maze. Although large-scale neurogene¬ 
sis stops around the time of birth, a few parts of the brain continue 
to add neuronal stem cells throughout life. They are the dentate 
('tooth-like') gyrus of the hippocampus (above) and the subven- 
tricular zone (located below the lateral ventricles). New brain cells 
may enable greater plasticity. Source : Kempermann, 2008. 
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That fact suggests a limited number of genetic 
changes, but the particular genes responsible are not 
known. Indeed, there are no other mammalian ana¬ 
logs for the forward fields of the human frontal lobes. 
However, primates have many brain similarities to 
other mammals, even ones as small as mice, as shown 
by the matched colors - the light and dark gray, light 
and dark purple, red and blue zones of Figure 16.8. 
The visible core of the brain is conserved among 
mammals, but the neocortex and frontal lobes have 
expanded dramatically. 

Homo sapiens is marked by a large forward expan¬ 
sion of the cerebral cortex (the prefrontal lobes), which 
has been called The organ of civilization' (Chapter 12). 
It appears to be distinctive among all living species. 
The human neocortex is about 1000 times larger than 
the neocortex of the earliest mammals. Other large¬ 
brained mammals, like whales and elephants, also have 
expanded cortices but proportionally somewhat smaller 
prefrontal lobes. The most prominent frontal functions 
are language production, social behavior, and high- 
level executive control. Language and symbolism are 
distinctive features of human culture. Executive con¬ 
trol allows us to carry out plans and work with others. 
Social cognition is a key to a cooperative society but also 
to out-group aggression, and both the serotonin system 
and monoamine oxidase enzymes (MAO) have been 


associated with aggression. Aggression may be adaptive 
for hunter-gatherers, who must be prepared to defend 
themselves or to kill prey or competitors, but it may 
become less functional in highly organized social groups. 


3*0 GENE EXPRESSION AND 
REGULATION 

All human cells have a complete copy of our species 
genome in their nuclei, in the form of a long, linear DNA 
code, twisted into the well-known double helix. The 
genetic code is expressed in proteins that define, develop, 
regulate, and control the phenotype. In each cell nucleus 
DNA is transcribed into messenger RNAs (mRNAs), 
which are then translated into proteins. 

A formal definition states that a gene is 'a locatable 
region of genomic sequence, corresponding to a unit 
of inheritance, which is associated with regulatory 
regions, transcribed regions, and/or other functional 
sequence regions' (Pearson, 2006). There are far more 
gene-regulatory regions than previously thought. 
Simple gene-to-protein translation is rare. The genome 
is an intricate network, not a one-to-one dictionary. 

In some cases, like simple Mendelian inheritance in 
fruit flies, we can see how DNA expresses the proteins 
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FIGURE 16.8 Of primates and mice. Even though the brains of mice and primates differ, core mammalian areas 
are remarkably similar (see tinted zones). The hippocampal region is highly conserved among mammals and even 
birds, and may have close homologies in reptiles and fish. The visible outer brain in humans is mostly cerebral cortex 
and cerebellum (underneath). In rodents the neocortex looks like a flat layer on top of the olfactory brain. The neo¬ 
cortex is in fact a flat, six-layered structure, but in humans (and whales) it is so large that we see it only as a deeply 
folded shape. Source : Eichenbaum, 2003. 
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that code for a phenotypic feature like eye color. But 
most genes work more broadly and more interactively 
Nevertheless, DNA still is expressed, directly or indi¬ 
rectly, in the proteome, a huge collection of species-specific 
active proteins and their basic parts, the amino acids , 
which have an amine group, NH 2 , and a carboxylate, 
COOH. Proteins are the basic constituents of life, along 
with oils, fats, minerals, and some small molecules. 


3*1 Information processing in the cell 

The Central Dogma has three information-carrying mol¬ 
ecules, and three basic steps between them (Figure 16.9). 

1 DNA-to-DNA replication. 

The first information transfer comes with reproduc¬ 
tion, where the DNA from the father and mother are 
passed to their child. This DNA-to-DNA transfer is 
called replication, and traditionally it has been viewed 
as a protected code that is almost never altered, with 



FIGURE 16.9 Gene expression: the Central Dogma. The 
assembly line of life. The Central Dogma of molecular biology 
claims that molecular causality flows from genes to messenger RNA 
to body proteins, but not the other way around. It has been chal¬ 
lenged by a number of interesting discoveries, but we begin with 
the Central Dogma and touch on exceptions later. Source: Baars. 


the exception of random mutations. Most mutations 
are believed to be dysfunctional and tend to die out. 
That it why it is thought, for example, that species like 
sharks have remained largely unchanged over hun¬ 
dreds of millions of years. Since all physical systems 
lose structure and information over time, according 
to the thermodynamic law of entropy, achieving that 
kind of constancy over hundreds of millions of years 
is an extraordinary feat of the genetic molecules. 

DNA-DNA replication is not believed to change the 
code, except in the case of mutations and genetic drift, 
mechanisms that allow for adaptation when a species 
encounters environmental pressures affecting its sur¬ 
vival and reproduction. 

2 Transcription: DNA to messenger RNA. 

DNA, the blueprint of life, is copied into messenger 
RNA by a process called transcription. We can still 
think of this as a copying process. However, molecules 
called Transcription factors' have a powerful impact 
on this stage of DNA expression. 

3 Translation: mRNA to proteins. 

All of life involves proteins, chains of amino acids. 
Muscle and organ tissue is mostly protein. The control 
system of every cell, DNA and RNA, enzymes, and 
transcription factors are proteins (Figure 16.10). They 
provide the major structural and functional elements 
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FIGURE 16.10 DNA: simple units combine in endless ways. 
Considering the complexity of living organisms the basic elements of 
the genetic code are amazingly simple. Four amino acids make up the 
base pairs that constitute the letters of the genetic code for all organ¬ 
isms (G = Guanine, T = Thymine, C = Cytosine, and A = Adenosine). 
The figure illustrates chromosomes in their characteristic x-shaped 
form at the control center of every living cell. The chromosomes con¬ 
sist of DNA surrounded by molecules that can turn on and off specific 
sections of the DNA. Source : U.S. Dept, of Energy. 
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of the body. The translation from RNA to proteins is 
the final step in the chemical factory of the cell. 

3.2 Higher-order DNA regulates other DNA 

By far most DNA is 'noncoding' and is not directly 
expressed in active proteins. About 3% of the noncod¬ 
ing DNA seems to perform regulatory functions, con¬ 
trolling other stretches of DNA. Examples include the 
body-plan genes, genes that control the sequence of 
brain development, and those that control the circa¬ 
dian cycle. We can think of those as spatial and tem¬ 
poral plans. 

Homeobox genes, called Hox, are highly conserved 
among vertebrates. That is why mice and humans 
and even salamanders have a long body axis, with 
two hindlegs and two forelegs, a longitudinal diges¬ 
tive tract, a spinal cord with a brain on top. Body 
plan genes control other genes that guide the growth 
of limbs and organs. The point is that Hox is easy to 
understand as a set of higher-order genes that con¬ 
trol and interact with many local genes. Figure 16.11 
shows how a set of Hox genes define the anatomy 
of the upper body, neck, and head in a wide variety 
of organisms. Figure 16.12 shows how some internal 



FIGURE 16.11 Hox genes determine tail-to-head positioning. 
Nature observers have long realized that there are great similarities 
even between very different organisms. One of those is the bilateral 
symmetry of humans, mammals, reptiles, and fish. We now know 
that such basic body plans are controlled by Hox genes, which are 
"ultra-conserved/ with an extraordinarily wide distribution in the 
animal kingdom. There are of course other body plans as well, as in 
squid and crabs. Source : Chambers and McGonnell, 2002. 


regions of the brain are also regulated by members of 
the Hox gene family. 

Hox genes work via transcription factors , proteins 
that affect the transcription of DNA to RNA. This is 
one way for the flow of causality to go in a nonclas- 
sical direction. Many steps on the classical pathway 
from DNA to active proteins can be modified by regu¬ 
latory stretches of DNA. 

In order to develop the distinctively human large 
frontal lobes the DNA body plan must know where to 
start adding neurons, and at which stage in the devel¬ 
opmental sequence. A plumber may need to under¬ 
stand the overall house plan to know where to put 
a water faucet. The growth of cortex also has to take 
place in the context of the overall body plan. The body 
plan is therefore one kind of 'higher-level' regulatory 
plan for features of the brain. 

Failures of regulatory genes can be serious. Some 
brain diseases are developmental errors, flaws in 
the moment-to-moment genetic control of neuro¬ 
nal growth. On the other side, regulatory genes also 
present potentially valuable targets for medical drugs. 
We will see some examples of both. 
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FIGURE 16.12 Hox and brain anatomy. Hox body plan genes 
also control the development of some brain regions. Notice the three- 
dimensional brain atlas (a), and the color-coded atlas of mouse brain 
regions (b). The large panel shows the levels of molecular expression 
of Hox genes (and some others) in the mouse brain. Source : Zapala 
et al., 2005. Reprinted in Zapala, Barlow, and Hovattaa, 2009. 
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3*3 An error in brain development 

To understand the development of the neocortex we 
can study genes that regulate growth, and also those 
that may underlie a failure to grow. Currently about 
half a dozen genes appear to regulate the size of 
neocortex; however, there may well be more. A gene 
called microcephalin was discovered in families at risk 
for microcephaly, in which the growing brain is dis¬ 
placed by cerebrospinal fluid. Somewhat different 
birth defects are called anencephaly and hydranenceph- 
aly. One lesson from growth failures of the neocortex 
is the extraordinary adaptability of the infant brain 
even under adverse circumstances. 

The condition called hydranencephaly is perhaps 
the most baffling example. Babies with that condi¬ 
tion appear to have no neocortex at all (Figure 16.13). 
It is now believed that part of the cortex may grow 
in utero, but that for unknown reasons it breaks down 
and is absorbed before birth. The early cortex may be 
able to transfer some of its capabilities to neighboring 
tissues before it is lost. Although most babies with this 
condition die in the first year, some of them live longer 


and seem to show surprising behavioral capacities 
(Figure 16.14). 

Hydranencephalic babies may seem normal dur¬ 
ing the first few months and bond with parents and 
siblings. When the symptoms finally become obvious, 
it may be difficult for families to come to terms with 
their loss. Parents have formed support groups that 
give more information on the web. 

3.4 Development-controlling genes 

One way evolution makes a quick fix in the face of 
a sudden environmental change is by changing a 
developmental stage. For example, anthropologists 
believe that the human skeleton is more gracile than 
Neanderthals were; that is, it is not as big-boned, 
robust, and well-protected. One way to make a robust 
body more gracile is by expanding its childhood and 
adolescence. Such 'juvenilization' occurs often. For 
example, cats and dogs may have become domesticated 
by way of selective juvenilization, since young mam¬ 
mals become more easily attached to mother substitutes 



FIGURE 16.13 Missing cortex in a hydranencephalic child. An MRI scan of a baby with hydranencephaly. 
The dark upper half of the cranium in the X-ray photos is filled only with fluid. The cerebellum and brainstem 
appear to be intact. The basal brain is responsible for basic sleep and arousal, and the child may seem to seem 
to have normal sleep-waking cycles for some time after birth. The surprise is that babies with hydranencephaly 
may behave somewhat normally, and only reveal visible deficits over a period of months. The cause of this dis¬ 
order is not currently understood. Source : Merker, 2007. 
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FIGURE 16.14 Some children may appear relatively normal without a cortex. Young children born with hydranencephaly (the little girl 
on the right) still show many behaviors that look relatively normal and that encourage bonding with caregivers. Source : Merker, 2007. 


than adults do. From the viewpoint of our pets humans 
may be substitute mothers and pack leaders. 

Juvenilization is another example of a regulatory 
change in DNA, a higher-level plan that modifies 
other stretches of DNA. Regulatory genes may be 
a promising target for drug design, since they exer¬ 
cise such extensive control. Changing developmental 
stages can allow a whole suite of traits to be altered 
in a short time. The interaction between evolution and 
development has become a vigorous research fron¬ 
tier, sometimes called 'evo-devo' for 'evolution and 
development.' 

FOXP2 is such a developmental gene. In a wide 
range of species it seems to be needed for develop¬ 
ing the lungs and vocal muscles. DNA extracted from 
Neanderthal bones shows the same version of FOXP2 
that humans have. However, Neanderthals apparently 
had less developed symbolic and language capacities. 
The FOXP2 story remains unsettled. 

3*5 Gene programming: beyond the 
central dogma 

If you stay up all night studying for an exam you are 
mildly affecting the genetic program that keeps your 
body in synch with the light cycle. If you do that 
many times, some long-term change can take place. 
Gene expression in human brain cells can be changed, 
within limits, by selectively turning stretches of DNA 
on or off (Figure 16.15 and 16.16). Such 'epigenetic' 


flow of control runs 'on top of' (epi-) the flow of nor¬ 
mal top-down causation. 

The discovery of epigenetic programs shows that 
the network of causation is even more complex than 
scientists used to think. For example, the environment 
of the fetus during pregnancy is shaped by the moth¬ 
er's health, alcohol and drug intake, and stress levels. 
Even the quality of maternal care can change epige¬ 
netic regulation. Fetal alcohol syndrome is an epigenetic 
disorder in which a pregnant woman's alcohol intake 
acts to retard and distort fetal brain development. Not 
all epigenetic disorders are due to substance abuse, of 
course. Environmental toxins are also believed to have 
epigenetic effects over the human lifespan. Some sci¬ 
entists think that autism may be an epigenetic disorder, 
neither completely genetic nor environmental, but a 
result of some unknown gene-environment interaction. 

In sum, molecular biology has become less of a one¬ 
way street and more of a network of genes, gene prod¬ 
ucts, and environmental inputs. 

3*6 The environment can restructure 
chromatin in the cell nucleus 

Chromosomes consist of long stretches of DNA, 
wrapped into tight bundles in the cell nucleus and 
surrounded by a host of support molecules. Humans 
have 23 chromosome pairs with an estimated 30 000 
genes. DNA wraps around histone spheres, proteins 
that give the chromosome a more compact structure. 
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FIGURE 16.15 Epigenetics: environmental shaping of gene 
expression. In what is sometimes called the Third genetic revolu¬ 
tion' biologists have discovered that the flow of control in the cell 
does not only go downward from DNA to RNA and then to active 
proteins. A number of environmental conditions can silence 
stretches of DNA, or activate them, or change the relationship 
between DNA, RNA, and the final protein product. Transcriptional 
regulation occurs at the level of the transcription between DNA and 
RNA. Source: Baars. 



FIGURE 16.16 The epigenetic code can switch gene expres¬ 
sion on and off. Epigenetic influences can silence or activate DNA 
stretches. Histone spheres (light green) act as spools to wind long 
strands of DNA (blue) into compact bundles, thereby packing 
almost two meters of DNA into a hundred micrometers of chromo¬ 
somes in the cell nucleus. To turn stretches of DNA on or off, parts 
of it must be unwound from their histone spools. Source: National 
Institute of General Medical Sciences, 2009. 


like a ball of yarn; together, they form a substance 
called chromatin. 

Cocaine and other external agents can alter the con¬ 
figurations of these proteins. Depending on the type 
of chemical change, chromatin either bunches up or 
stretches out, activating or silencing genes along the 
DNA sequence. Chromatin reshaping seems to underlie 
healthy adaptations such as learning and memory as 
well as disease processes - including cancer, seizures, 
schizophrenia, and depression. Medicines like antide¬ 
pressants also seem to reshape chromatin, in addition 
to other molecular actions. 

In rodents, maternal touching and licking has the 
effect of unwinding neuronal DNA strands from his¬ 
tone spools, thereby allowing RNA polymerase to ena¬ 
ble DNA transcription to mRNA followed by protein 
translation. Lower maternal care will tend to 'silence' 
DNA by keeping it tightly wrapped on histone spools. 
The chemical factors involved are TSA (trichostatin A) 
and the methyl donor methionine. In Figure 16.17, the 
green tags marked A refer to acetylated histone tails. 
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FIGURE 16.17 Maternal care can modify DNA expression. 
Maternal grooming of rodent pups can facilitate or inhibit DNA 
expression. Source: Champagne and Curley, 2009. 

which allow the unwinding and exposure of DNA. 
At the right end of the diagram the tag labeled RNAp 
refers to RNA polymerase, the molecule that enables 
DNA transcription via messenger RNA. 
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3.7 Learning as an epigenetic process 

As we will see, epigenetics now has become crucial to 
the study of learning. Neurotransmitters alter the gene 
expression of their cells of origin and their target cells 
as well. Since DNA acts as the control program for the 
production of molecules, including neurotransmit¬ 
ters, this makes sense for the presynaptic cell. But it is 
equally true for the postsynaptic cell, and for neigh¬ 
boring glial cells as well. These points are covered in 
more detail in this chapter's section on learning. 


4*0 NEURONS AND GLIA AS 
SIGNALING CELLS 

All living cells share fundamental features: a nucleus 
with genetic control machinery, cellular organelles 
like the energy-making mitochondria, basic metabolic 
digestion of oxygen and glucose, and waste disposal 
of toxins, all held together by a two-layer lipid mem¬ 
brane. All cells communicate by way of signaling 
molecules that can pass via the membrane under well- 
controlled conditions. Neurons are like other animal 
cells in these respects; but they also are specialized in 
electrochemical signaling and computation. 

To accomplish the job of neurocomputation and 
signaling, neurons are the most highly branched cell 
type, serving both as cables and as switching gates, 
perhaps even as small computers. Neurons are dis¬ 
tinctive in being able to send fast electrochemical sig¬ 
nals along their long axons, by converting the voltage 
difference across the polarized membrane into action 
potentials (see Chapters 1 and 3). Most importantly, 
neurons synapse onto others, either through the classi¬ 
cal synapse that was identified decades ago, or by way 
of a variety of nonclassical signaling pathways. 

Recently glial cells, the 'glue' that was thought to 
hold neurons in place, have been found to be bigger 
players in the brain signaling drama. Glial cells con¬ 
stitute about half of the total mass of the brain. That 
suggests important functions. Traditionally they have 
been billed as the supportive players for the biggest 
actors on stage, the neurons, but glial cells can seize 
the spotlight as well. Glia do their own kinds of sig¬ 
naling, using calcium waves that propagate more 
slowly than neurons fire their axonal spikes. But neu¬ 
roglia also interact with faster neurons via neurotrans¬ 
mitter molecules, and they are the only place in the 
brain where the Big Two neurotransmitters actually 
are made (GABA and glutamate). 


4*1 Neurons and synapses as on/off switches 

A neuron can be considered as a switch, a biologi¬ 
cal version of the electronic switches in computers. 
The axon either fires or it does not, flipping between 
'send' and 'don't send.' In the same way, the synapse 
acts as a switch, either allowing chemical transmission 
or blocking it. The input branches of each nerve, the 
dendrites, can be viewed as integrators, adding up 
the voltage inputs from tens of thousands of incoming 
signals over a very brief period of time. These graded 
voltages act somewhat like an analogue computer. 
The flow of information can therefore be stopped, 
modified, or amplified in each nerve and at each syn¬ 
apse. Although brains are not identical to computers, 
they are similar in allowing a very large set of switch¬ 
like elements to detect, transform, analyze, filter, test, 
store, retrieve, plan, and control information. Since the 
brain has about a hundred billion nerves, and trillions 
of synapses, it can be thought of as a massive collec¬ 
tion of information processors, somewhat like the 
World Wide Web. 

It is just as useful to think about the brain as a giant 
adaptive organ. In some ways it resembles a biologi¬ 
cal species, constantly engaging in massive synaptic 
reproduction, variation, and selection. But rather than 
taking millennia, the brain does its Darwinian selec¬ 
tion over second, minutes, and hours. 

Notice that all processes in Figure 16.18 are self- 
regulated to ensure that concentrations of biomolecules 
stay within tight limits. That is the role of autorecep¬ 
tors for controlling synthesis and release, and therefore 
the resulting neuronal firing rate. Transporter mole¬ 
cules recycle neurotransmitter molecules back to their 
sources to be reused. ('Vesicular transporter' refers to 
molecules being recycled to the vesicles, small bubble¬ 
like capsules that ferry neurotransmitter molecules to 
the synapse.) Metabolizing enzymes may chop neu¬ 
rotransmitter molecules into pieces to inactivate them 
and recycle their parts. Notice that membrane recep¬ 
tors look like tiny threads, which wind in and out of 
the membrane. That symbol serves as a reminder that 
membrane receptors are proteins, long chains of amino 
acids, which can be classified by the number of times 
they are threaded into the membrane. 

The crucial question is how messenger molecules 
are made, ferried to the synapse, and released and how 
they are recognized - or blocked - at the postsynaptic 
cell membrane. Finally, the target cell needs to convert 
incoming molecular signals into new membrane volt¬ 
ages, ultimately sending another spike whizzing down 
its own axon (Figure 16.19). 
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transporter 

FIGURE 16.18 A basic synapse. Two neurons separated by a synapse, with neurotransmitter mol¬ 
ecules released from the first cell and diffusing across the synaptic cleft to the second cell. It is conven¬ 
ient to call the first neuron presynaptic and the second one postsynaptic. The glial cell on top performs 
support functions, but neuroglia are now known to do their own information processing as well. It is 
therefore useful to think of synapses as involving three cells: presynaptic, postsynaptic, and glial sup¬ 
port. Source : Byrne and Roberts, 2004. 



FIGURE 16.19 Gap junctions make direct connections. "Gap junctions' are direct electri¬ 
cal connections between cells. Most GABA-ergic junctions in cortex are gap junctions, which 
allow for faster transmission and therefore more precise synchrony among large numbers of 
cortical neurons. Source : Cunningham and LeBeau, in Squire, 2009. 

4*2 Chemical self-regulation i i • „ ^ _ 

° molecule is generally toxic, and can kill nerve cells. For 

Brain chemicals are closely regulated. If the synapse that reason there are pervasive negative feedback loops in 

has too little of a neurotransmitter, its message will not any chemical pathway, much like tiny thermostats that 

be sent. On the other hand, too much of the signaling keep the quantity of any molecule within safe limits. 
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In the synapse itself there are many mechanisms 
to remove excess molecules of neurotransmitters - by 
turning down the rate of synaptic release, letting mol¬ 
ecules diffuse away from the synapse, transporting 
them back to the originating cell, neutralizing them 
chemically, recycling them, or allowing glial support 
cells to vacuum up the excess. Cleaning up the syn¬ 
apse is a vital function. Synapses that become satu¬ 
rated with a neurotransmitter may cease working, and 
their neurons may die off. A good example is nitric 
oxide, which acts as a neurotransmitter in many syn¬ 
apses - that is how the drug Viagra works - but which 
turns into a toxic gas if it does not dissipate quickly. 
Nitric oxide diffuses away from the synapse within 
milliseconds if the amount is small. 

A good example of chemical self-regulation in 
the brain is the existence of auto-receptors. It is vital 
for neurons to control their own secretions - partly 
because they can end up poisoning themselves, and 
partly because too much of a transmitter can fill the 
synapse with signaling molecules so that it never 
stops sending the same signal over and over again. If 
you hear the same message repeated endless times it 
loses information. The same thing is true of the brain. 
Neural signaling is only informative if it stops and 
starts in a meaningful way. For those reasons every 
neuron must regulate its own secretions of signaling 
molecules. 


43 Membranes, ion channels, and 
axonal spikes 

Here is a brief review of our previous discussions of 
cells and molecules in Chapters 1 and 3. 

All animal cells act like tiny electrical batteries, 
maintaining a voltage difference across their mem¬ 
branes, on the order of —70 to 100 millivolts (mV) 
(Figure 16.20). That voltage difference is maintained 
by active membrane pumps, which push positive ions 
outside of the cell, and negative ones inside. It requires 
metabolic energy to sustain that difference in electri¬ 
cal charge, and to run other cell functions, so that all 
nerve cells must constantly consume energy. That is 
why the brain is such an energy-hungry organ. 

Unlike other cells, neurons can convert their mem¬ 
brane voltage into electrochemical signals, as shown 
in Figure 16.21. The input branches of a neuron accept 
signals from neighboring cells, which may change the 
voltage on the dendritic input branches. When these 
voltage changes add up to about 70 mV near a point at 
the beginning of the axon, they can Tire a spike' - a fast 




FIGURE 16.20 Basics of neuronal signaling. All cells main¬ 
tain a voltage difference across their membranes, but only neurons 
utilize that voltage to propagate an axonal spike. The figure shows 
how an imbalance between positive ions outside the cell and nega¬ 
tive ones inside create a voltage gradient, which must be maintained 
by constantly pumping ions in the opposite direction. When a spike 
is fired the charge difference is momentarily lost, and must be rees¬ 
tablished by pumping ions again. The neuron is therefore constantly 
utilizing energy to pump ions. Source: Shepherd, 2003, in Squire 
et al. Original from Spencer and Kandel, 1961. 

wave of depolarization in the axonal membrane. The 
ions (charged particles) that have been pumped to the 
different sides of the membrane suddenly are released 
as the membrane opens up. This sends a wave of 
electrochemical activity toward the synapses. The 
axonal membrane voltage drops locally from —70 mV 
to about +30 mV, but the voltage is restored to base¬ 
line within milliseconds. The result is a fast wave of 
depolarization shooting down the axon from the cell 
body to the synaptic buttons. 
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FIGURE 16.21 The cell membrane pumps ions. Lipid bilay¬ 
ers (two layers of molecules) make up cell membranes, pointing 
toward each other like two matchsticks. The head of each lipid is 
hydrophilic (water-loving), and the tails are hydrophobic (water- 
avoidant). As a result, the molecules line up spontaneously in a 
nanometer-sheet, with the water-loving lipid heads pointing to 
the surrounding watery medium (inside and outside), while the 
hydrophobic ends point to each other. Cell membranes are essen¬ 
tially bubbles made of lipid bilayers, protecting internal processes 
by means of a flexible barrier only two molecules thick. Source : 
McCormick, in Byrne and Roberts, 2009. 


This system can be turned on and off, to make a 
switch, or perhaps a more complex computational ele¬ 
ment. To excite a neuron we need to depolarize the 
membrane. That can be done by injecting an electrical 
current into the cell to reverse its polarity, in much the 
way you might switch the two electrodes on a battery. 
Or we can introduce a neurotransmitter into the syn¬ 
apse to signal the postsynaptic neuron to depolarize. 
The effect is the same. By depolarizing the membrane 
we send a spike down the long axon, to trigger the 
release of neurotransmitter into the synapse. 

That is the classical model of the excitatory neuron. 
However, we can also inhibit the postsynaptic neu¬ 
ron by /zyperpolarizing its membrane. That is, we can 


raise the voltage difference across the membrane. The 
hyperpolarized membrane takes more energy to be 
depolarized. As a result, we lower the probability that 
the postsynaptic neuron will fire. 

It is easier to see this excitation and inhibition in 
terms of changes in the spontaneous firing rate of 
the neuron. Under normal conditions neurons fire at 
a 'base rate/ even if they do not receive stimulation. 
A hypothetical base rate might be one spike per sec¬ 
ond. If the neuron is stimulated by an excitatory neu¬ 
rotransmitter, the rate of firing rises above 1Hz. But 
if the neuron receives an inhibitory neurotransmitter 
the base firing rate may drop to zero. As mentioned 
earlier, the most common inhibitory neurotransmitter 
is GABA, while glutamate is the most common excita¬ 
tory neurotransmitter. 

The lipid bilayer forms spontaneously when lipid 
molecules are put in water. Lipid molecules have one 
end that is attracted to water (hydrophilic) and another 
end that is repelled by water (hydrophobic). Lipids 
therefore couple with their hydrophobic ends toward 
each other, and hydrophilic ends toward the watery 
medium. Two-layered membranes therefore form 
spontaneously, and in the case of cells, become spheri¬ 
cal bubbles. The lipid bilayer is both flexible and highly 
stable, and is highly conserved among living organ¬ 
isms. Proteins embedded in the two-layered membrane 
form ion channels and pumps, chemical receptors and 
enzymes, and many other structures to enable hun¬ 
dreds of different processes. 

In a sense, all the ion channels we have described in 
running neurons are voltage-gated. That is, we think 
of the cell membrane as maintaining a voltage differ¬ 
ence between the inside and the outside of the cell. It 
takes active energy-using processes to make that hap¬ 
pen, specifically the ion pumps that are embedded in 
the cell membrane. We already know that the electrical 
polarity across the membrane breaks down for some 
milliseconds, every time a neuron is electrically stimu¬ 
lated, or whenever it generates a change in voltage due 
to rapid depolarization. Thus electrical events generate 
a local chemical ion exchange across the membrane, 
and vice versa, in a very rapidly cycling loop. 

Figure 16.22 shows a voltage-gated ion chan¬ 
nel at a detailed level. Each type of ion channel can 
be described chemically in terms of the number of 
long-chain protein loops that thread to the inside and 
outside of the membrane. These are called transmem¬ 
brane segments, with labeled subunits. A fairly simple 
description like this helps to classify large numbers of 
membrane-embedded proteins, including ion chan¬ 
nels and receptors. 
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FIGURE 16.22 A voltage-gated ion channel. A voltage-gated sodium ion channel (Na+), with its char¬ 
acteristic long-chain protein looping in and out of the cell membrane with six transmembrane segments, each 
one repeated four times (DI to DIV). Its two subunits are labeled alpha and beta, and the ends of the long 
protein thread are labeled N and C (for nitrogen and carbon). Source : Squire, 2009 [Encyclopedia of NS]. 


5.0 SYNAPTIC TRANSMISSION: FROM 
PRODUCTION TO CLEAN-UP 

Messenger molecules in the brain are of two kinds: 
transmitters and modulators. We can define neuro¬ 
transmitters as having very local effects in the synapse 
between two neurons, although there might be billions 
of such neurons and synapses secreted by presynaptic 
cells. Neuromodulators , on the other hand, are produced 
by quite small clumps of cell bodies below the cortex, 
and are spread very widely. These cells - typically just 
a few thousand - project their branches through sizable 
parts of the brain. Where they terminate, they 'spray' 
their messenger molecules. Thus neuromodulators can 
change large regions of the brain, while neurotransmit¬ 
ters act locally in nano-level synapses (see Section 6.0). 

5.1 The big two: glutamate and GABA 

Glutamate and GABA are the two most common neu¬ 
rotransmitters in the brain. Ninety percent of the 
neurons in cortex use glutamate, the primary excita¬ 
tory neurotransmitter, which increases the probability 
of the next neuron firing when a precise number of 
molecules are released into the synapse. In the human 
brain glutamate is used in most of the long pyramidal 
neurons of the cortex and deeper brain structures. It is 
used at most synapses that are modifiable - capable of 
increasing or decreasing in strength, thereby making 
learning possible. 

In contrast, GABA is the major inhibitory neurotrans¬ 
mitter in cortex. Inhibitory synapses decrease the prob¬ 
ability of firing of the next, postsynaptic cell. GABA 
is used in the bushy interneurons that surround the 
long pyramidal cells of the cortex, and is considered 


to be the molecule for 'sculpting' and regulating con¬ 
stant excitatory activity in the cortex. 

We do not want all our excitatory synapses on all 
the time. The brain might go into higher and higher 
positive feedback loops, like a runaway nuclear power 
reactor. Cortex would be overloaded, as it does in the 
case of epileptic seizures. An excess of glutamate is 
toxic, called excitotoxicity. Much of the damage due 
to strokes is not direct, but the result of toxicity from 
an excess release of glutamate. It is just as if an auto¬ 
mobile catches on fire and the fuel tank explodes: the 
secondary explosion can be more destructive than the 
original fire. Neurotransmitters are useful only in tiny, 
well-controlled amounts. 

Glutamate (abbreviated Glu) is also a good example 
of the reutilization of preexisting molecules for neural 
transmission. Glu is one of the dozen essential amino 
acids that make up animal proteins; these are the ones 
that cannot be manufactured in the body from other 
amino acids, and therefore must come from food. The 
brain therefore borrowed glutamine, one of the basic 
building blocks of life, to use as its primary excitatory 
neurotransmitter. 

We can also taste glutamate in food, as Japanese 
researchers discovered in 1907 when they analyzed 
the residue of soy sauce. The taste of glutamate is 
one of the five basic flavors for which we have taste 
receptors, the subtle flavor called umami. The taste of 
glutamate tells us whether food is edible and fresh - 
essential information for hunter-gatherers trying to 
survive in the natural world. 

We can think of this three-cell system as a tiny fac¬ 
tory made especially to produce a tiny, controlled 
quantity of the neurotransmitter glutamate, trans¬ 
port it to the synapse using small bubbles called vesi¬ 
cles, and squirt each one into the gap to diffuse to the 
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FIGURE 16.23 Glutamate signaling needs three cells. A system of three cells that cooperate to support glutamater¬ 
gic signaling. On the right side, notice a blood capillary carrying glucose and oxygen to the astrocyte cell and to the two 
signaling neurons to supply their activities. Glucose also provides raw material for the production of glutamate. (Vm is 
the membrane voltage of the top neuron, showing all-or-none spikes, which ultimately triggers the release of the neuro¬ 
transmitter at the synapse.) Source : Magistretti in Squire et al., 2008. 


postsynaptic cell (Figure 16.23). The tiny oval in 
the top cell is a mitochondrion, an energy-producing 
organelle. The whole three-cell system is fed directly 
by glucose and oxygen diffused across the membranes 
from the blood capillary on the right. As mentioned in 
the caption, glucose is used for energy and also for the 
raw material to make glutamate, the neurotransmitter. 

Notice that the target cell (postsynaptic) has two 
kinds of glutamate receptors, which we will discuss 
later. Metabotropic receptors utilize the existing meta¬ 
bolic machinery of the target cell. Ionotropic receptors 
activate the same kinds of ion channels we have seen 
already: sodium, potassium, and calcium channels. 

Finally, the astrocyte (star-shaped) cell in the mid¬ 
dle is crucial to the whole nano-factory. It takes in glu¬ 
cose fuel, breaks it down (glycolysis), and converts 
ATP into ADP for usable energy inside its own mito¬ 
chondria, squirts glutamine into the presynaptic cell 
to make glutamate, and soaks up the excess glutamate 
from the synapse after it is released. The last step is 
crucial, because glutamate is toxic if it is allowed to 
dwell too long outside of the cellular machinery. In 
fact, as we will see, glutamate toxicity is believed to 


contribute to serious brain disorders. (It is also called 
excitotoxicity, because glutamate is the most popular 
excitatory neurotransmitter in the brain.) 

If this seems like a lot of trouble to go through just 
to squeeze out some glutamate in your brain, bear 
in mind that neurons and glutamate transmission 
are a very ancient and biologically perfected signal¬ 
ing system. The earliest mammals used the identi¬ 
cal molecular machinery 200 million years ago. Even 
the round worm C. elegans has the same machinery. 
Glutamatergic signaling is exquisitely precise in its 
timing, and provided the neurotransmitter is cleaned 
up quickly, it does not leave toxins in the fluid sur¬ 
rounding our brain cells. Almost all biochemical 
processes produce some toxins (especially oxidation 
products), and these are very harmful over time. 

5.2 GABA: the major inhibitory transmitter 

Excitatory neurons are needed to make things hap¬ 
pen, but if the brain only had excitation it would go 
into rapid overload, with every glutaminergic neuron 
exciting every other one. Inhibitory neurons are believed 
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FIGURE 16.24 A GABA synapse. Notice that glutamine-related molecules are used in 
the production of GABA. A transporter is an enzyme that moves a neurotransmitter molecule 
back to its originating cell, to be reused. As before, autoreceptors play a role in self-regulation 
of chemical concentrations in the synapse. Source : Deutch and Roth, in Squire et al., 2008. 


to balance excitatory ones to maintain overall homeosta¬ 
sis in activity levels. Excitatory and inhibitory cells also 
interact to generate regular rhythms, the most important 
fast interregional signaling system in the brain. 

The glutamate pathway also manufactures the major 
inhibitory neurotransmitter, GABA (for gamma amino- 
butyric acid) (Figure 16.24). When a neuron fires and 
releases GABA, the probability of the next neuron firing 
is decreased. GABA can have the effect of hyperpolarizing 
target neurons rather than depolarizing them, thereby 
making it less likely for the cell to be fired even if it also 
receives excitatory inputs. It is therefore the balance of 
inhibitory and excitatory inputs arriving at a target cell 
within a very short period of time (—10 milliseconds) that 
shapes its activity level at any moment (see Chapter 3). 

While the long pyramidal neurons of the cortex are 
excitatory, the smaller bushy interneurons that con¬ 
nect them locally are mostly inhibitory (Figure 16.25). 
Thus the great ocean of excitatory ripples and waves 
that keeps the brain constantly stirred up is inhibited 
locally by dense clumps of GABA-secreting cells. They 
are called GABA-ergic, and the excitatory ones are 
called glutamatergic. (The ending -ergic is from a Greek 
word for exerting force.) 



FIGURE 16.25 All cortical circuits involve both excitatory 
and inhibitory neurons. The pyramidal cells of the cortex are shown 
in light blue, and the inhibitory interneurons in red. Green neurons 
coming from outside of the system (perhaps from the visual tract) 
drive all the cortical pyramidal neurons. SI refers to 'smooth' neu¬ 
rons, which are also excitatory. Microcircuits like this are pervasive 
in the cortex, and make for local differences between anatomically 
similar regions of the brain. Source : Douglas and Martin, 2009. 
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Widespread inhibition is needed to regulate excita¬ 
tory neurons, and to create boundaries to the flow of 
excitatory transmission. To prevent excitatory transmis¬ 
sion from overloading the brain, there are numerous 
GABA synapses that regulate the levels of excitation. 

GABA is used at the great majority of fast inhibitory 
synapses in virtually every part of the brain. GABA is 
also a behavioral inhibitor. Most tranquilizers increase 
GABA-promoting brain activity (called GABA -ergic). 
The relaxing effects of alcohol are also due to its effect 
on the GABA system in cortex. But GABA has numer¬ 
ous functions, including ones that are more complex 
than just inhibiting the effects of excitatory synapses. 

We can therefore simplify things with some basic 
principles. Genes operate over many millions of years, 
over relatively brief human generations, and finally 
over days, hours, and minutes. The elaborate biochemi¬ 
cal dance of molecules makes use of preexisting biologi¬ 
cal molecules that have been made by living organisms 
for millions of years. Humans share certain fundamen¬ 
tals with an astonishing range of living and past life 
forms, but we have important differences as well. 

As we will see, those differences are due mainly 
to higher-level control of DNA, the regulatory DNA, 
rather than the DNA that is directly transcribed into 
messenger RNA and target proteins. We are not so dif¬ 
ferent from other animals in our basic protein building 
blocks. Rather, the human difference is in the 'higher- 
level' organization of our genome, which dictates the 
growth and operation of the expanded neocortex, 
especially the frontal lobes. Humans are a new mel¬ 
ody played on the basic musical notes of life. 

5.3 Neuroglia may also process information 

Neurons are the major signaling and information¬ 
processing cells of the brain. But they are not the only 
ones. We have already described the close cooperation 
between neurons and glial cells. There are many types 
of neurons and glia, but we will treat them as just two 
general types of signaling cells. 

There is ongoing debate about the exact role of glia in 
processing information. Neurotransmitters secreted by 
neurons induce calcium waves that propagate between 
glia (specifically astrocytes, the star-shaped kind of glia). 
Astrocytes also respond to visual stimuli with spatially 
specific receptive fields, like visual neurons. They are 
selectively sensitive to the orientation and spatial fre¬ 
quency of visual stimuli. Glial cells can act like neurons, 
therefore. But they are not identical, and there continues 
to be debate about their respective roles in basic cogni¬ 
tive tasks like sensory perception and memory. 



FIGURE 16.26 Glia use calcium waves for signaling. Glial 
cells can propagate calcium signals to each other by passive diffu¬ 
sion. These cells do not need synapses to communicate. Source : Haas 
and Kettenmann, in Squire, 2009. 

Because glia do not need synapses to signal to 
each other, it is believed that they allow for very fast 
spreading of activity. This is one type of gap junction. 
Figure 16.26 shows how a stimulus (the red lightning 
symbol) can trigger the release of ATP in a single 
glial cell. ATP evokes calcium waves, which diffuse 
through adjacent glial cells, generating widespread 
calcium waves. ATP and glutamate can also increase 
in the extracellular space, as shown. 

5.4 Production, release, and disposal of 
neurotransmitters 

Every cell is a chemical factory. Chemical production 
starts with raw materials, like the tryptophan in a glass 
of milk. Tryptophan is converted into serotonin via 
steps that are controlled by the DNA in the nucleus of 
the presynaptic neuron. Once the messenger chemical 
has been made, it is transported to the synapse, often 
using tiny bubble-like vesicles filled with transmitter 
molecules. These bubbles fuse with the membrane, 
and release their molecules into the synaptic cleft. 

Production of neurochemicals occurs via a clearly 
defined assembly line, a pathway or 'cascade' in 
which each step is enzymatically controlled within 
very precise limits. Figure 16.27 shows the catecho¬ 
lamine pathway, which results in such fundamentally 
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important molecules as dopamine, noradrenalin (nor¬ 
epinephrine), and adrenalin (epinephrine). It is possi¬ 
ble to influence the amount of molecules by enzymatic 
action, or by providing precursor chemicals like l- 
dopa. However, because each step is homeostatically 
controlled, adding more than the necessary amount of 
a precursor is not likely to affect the production cas¬ 
cade. By analogy, if an extra ten tons of steel is pro¬ 
vided to an automobile assembly line, beyond what is 
needed to actually make the cars, that steel cannot be 
utilized. It needs to be stored or disposed of. Because 
every step in the process is highly controlled, it can 
only be manipulated from the outside within certain 
limits. However, if there is a deficiency state, like a 
folic acid deficiency in pregnant mothers, providing 
the missing ingredient can be vitally important. 

5*5 Synaptic release 

Figure 16.28 shows a cartoon of a nerve cell with a clas¬ 
sical synapse, releasing messenger molecules into the 
synaptic gap. The molecules are free to diffuse from 
the presynaptic neuron and are detected by postsynaptic 
cell receptors. There are trillions of such synapses in 
the brain. Notice that the size of the voltage spikes is 


affected by recent activity. As the neurotransmitter is 
secreted into the synapse and therefore decreases in the 
cell, a fast-following spike can only release the remain¬ 
ing molecules. Again, there is a kind of bucket brigade 
in the flow of molecular production and release, where 
the timing of each bucket being passed depends on the 
timely hand-over of the previous bucket. 

In the classic model of the synapse small vesicles 
(yellow bubbles) transport neurotransmitter mole¬ 
cules down the axon to the synaptic surface, fuse with 
the membrane, and release their neurotransmitter into 
the synaptic gap. Other neurons release their neuro¬ 
transmitters in slightly different ways. Each vesicle 
may have on the order of several thousand molecules, 
again allowing for very precise control of its concen¬ 
tration both inside of the presynaptic cell, and in the 
synapse when it arrives there. 

The vesicles carry calcium-binding proteins, which 
are released as the vesicle touches the presynaptic 
membrane. It is these calcium-binding proteins that 
act as sensors to control the moment of neurotransmit¬ 
ter release from their vesicles into the target synapse. 
Calcium is therefore a major player in the delicately 
timed and precisely metered squirting of neurochemi¬ 
cals into the synapse (Figure 16.29). 
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FIGURE 16.27 Catecholamines: a production pathway for multiple neurotransmitters. The catecho¬ 
lamine pathway includes major neurochemicals like dopamine, noradrenalin (norepinephrine), and adrena¬ 
lin (epinephrine). Thus a single chemical pathway may produce multiple brain signaling molecules. One of 
these, L-dopa, was made synthetically as the first medication for Parkinson's disease, to increase the quantity 
of dopamine in the brain. Notice the similar structures of these molecules. Source : Baars, 2009. 
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5*6 Synapses and receptors are traffic 
control points 

Psychoactive drugs typically focus on one specific 
step in synaptic signaling. Thus L-dopa is prescribed 
for Parkinson's patients to increase a precursor for 
dopamine - an ingredient needed in the manufacture 



Copyright © 2002, Elsevier Science (USA). All rights reserved. 


FIGURE 16.28 Vesicles squeeze their molecules into the syn¬ 
apse. A simplified image of the release of neurotransmitter mol¬ 
ecules from vesicles (yellow bubbles) that transport the molecules 
from their site of manufacture, which may be in or near the nucleus, 
or in other organelles inside of the presynaptic neuron. Vesicles flow 
in a regulated sequence, often guided by microtubules, essentially 
submicroscopic tubes inside of the axon. On the left (A), the flow 
of vesicles leads to release of transmitter molecules in two equal 
bursts. In (B), the first burst depletes the terminal button, so that the 
second release is decreased. Source : Schwarz, in Squire et al., 2003. 


of the neuromodulator dopamine. Since Parkinson's 
disease involves the die-off of dopaminergic neurons 
in the substantia nigra, L-dopa will stave off its worst 
effects for some time. (Ultimately, the cells will have 
to be replaced, however, and much current research is 
focused on trying to implant dopaminergic cells in the 
substantia nigra (SN).) 

In other cases, the effort is to keep a neurotrans¬ 
mitter dwelling longer in the synapse, so that it can 
repeatedly stimulate the postsynaptic membrane. A 
famous example is the selective serotonin reuptake 
inhibitors (SSRIs) - psychoactive drugs that are effec¬ 
tive against depression and other disorders. Reuptake 
is the rather awkward term for recycling serotonin to 
the cell of origin, and if you can inhibit recycling you 
can keep the neurotransmitter around longer. 

GABA-ergic drugs like the benzodiazepines increase 
the amount of GABA in inhibitory synapses in the 
brain. And we should not forget the role of healthy 
diets, which supply all of the basic chemistry of the 
brain, mostly in the form of precursor chemicals, the 
raw materials from which the cells make all the spe¬ 
cific proteins and other products they need. 

A great deal of medical research is constantly trying 
to identify more traffic control points in the body and 
brain. Channel blockers work by inhibiting the flow 
of ions like calcium, sodium, and potassium across 
the cell membrane (important in cardiology as well 
as brain disorders). These are useful points to keep in 
mind. While the details are awesomely complex, the 
basic ideas are actually quite simple. 

There are many kinds of neurons and neuron-like 
signaling cells. Neuroglia were traditionally thought to 
be only support cells for neurons, but have since been 
found to do their own transmission and computation. 
They outnumber neurons by a factor of ten. (The word 
glia originated from the Greek word for the glue between 
neurons.) 



Nerve and cardiac excitability, Neurotransmitter release 
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FIGURE 16.29 Calcium triggers 

neurotransmitter release. Calcium 
has numerous signaling functions 
throughout the body. In the brain 
(and in neurally paced organs like 
the heart), calcium may be control¬ 
led by voltage rhythms. In neurons, 
a calcium influx may trigger the 
release of the presynaptic neuro¬ 
transmitter into the synapse. Source : 
Snutch, in Squire, 2009. 
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FIGURE 16.30 Receptors are long protein molecules embedded 
in the cell membrane. Long proteins thread their way across the cell 
membrane a countable number of times, and form typical membrane 
receptors and ion channels. Source : Squire et al, 2008. 


5*7 Receptors detect signaling molecules 

Receptors can be considered to be the locks into which 
the floating keys of neurotransmitters try to fit (Figure 
16.30). The lock-and-key metaphor is of course the 
favorite one for describing enzymes and their sub¬ 
strates. In fact, neuronal receptors are special cases of 
molecular receptors that are a universal feature of the 
chemistry of life, even in single-celled organisms that 
have no distinct nervous system at all. The target cell 
of a neurotransmitter is labeled the postsynaptic cell, 
while its cell of origin is called presynaptic. Those are 
obviously relative terms. In effect, we are looking at 
the synapse as a 'box/ with its input labeled as pres¬ 
ynaptic and its output as postsynaptic. If instead we 
consider the neuron to be the object of interest, what 
we consider to be the input and output would change. 
It is useful to keep this shifting focus in mind, to avoid 
confusion when reading about the microstructure of 
the brain. 

The glutamate receptor has been one of the basic 
test-beds for neurotransmission. Figure 16.31 shows 
the prototypical glutamate understanding of gluta- 
matergic (excitatory) neurotransmission. Notice that 
Glu has two different kinds of receptors, one that is 
ionotropic (ion-targeted) and a second kind, which is 
metabotropic, meaning that Glu locks into the existing 
metabolic machinery of the target cell. 

In the first case, we can think of the ionotropic 
receptor as just a set of ion channels that exchange 
sodium and potassium from one side of the membrane 
to the other, such as we see everywhere along the 
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FIGURE 16.31 Glutamate receptors: ionotropic and metabo¬ 
tropic. Glutamate actives two types of receptors, ionotropic and 
metabotropic. The ionotropic receptors open and close ion channels, 
just like the sodium and potassium channels that control the local 
voltage across the neuronal membrane everywhere else. On the 
right, metabotropic receptors trigger the existing metabolic machin¬ 
ery of the cell, which precedes that evolution of specialized nervous 
systems. G-protein-coupled receptors are also called "second mes¬ 
sengers/ Source : Forsythe and Barnes-Davies, 1997. 


membrane of the nerve cell. But in this case the ion 
channels are located in the postsynaptic membrane. 
Ion channels are selective pores, molecular gates that 
can open and close, either because of a change in volt¬ 
age or by way of a chemical receptor. 

We can take the analysis one level lower and notice 
that the ionotropic effect of glutamate can evoke two 
molecular machines, one involving NMDA and the 
other AMPA. Both of those substances are vital in 
the process of learning, and we will see their names 
again. The balance of NMDA and AMPA determines 
what kind of learning will take place at the target 
membrane of the glutamate synapse. NMDA is asso¬ 
ciated with long-term potentiation (LTP), making the 
synapse more excitable over time. AMPA is associated 
with long-term depression (LTD), which makes it less 
excitable. These terms therefore correspond roughly 
to excitation and inhibition, but not on a momentary 
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basis, rather in changing the enduring nature of the 
synapse. From a Hebbian point of view, these are the 
two logical changes we can expect in the strength of a 
synapse after learning. 

Metabotropic receptors can become very compli¬ 
cated, because they have evolved to take advantage 
of the preexisting metabolic ballet of the cell. But the 
basic idea is again fairly simple: it is that the neuro¬ 
transmitter glutamate locks into a receptor complex 
that triggers G proteins (G stands for the DNA base 
pair guanine). G proteins have been studied exten¬ 
sively as part of the basic biology of animal cells, both 
by way of the universal role of metabolism and of the 
genetic control machinery of DNA. G-protein-coupled 
receptors are also called 'second messengers/ because 
the first messenger (the neurotransmitter molecule) 
hands on the baton to the second one, and so on. 
Second messengers can evoke third messengers that 
even are more deeply embedded in the nuclear appa¬ 
ratus of the cell, and so on. 

G-protein-coupled receptors are important for 
very practical reasons. A surprisingly large number of 
medical drugs are believed to utilize this level of neu¬ 
rotransmission and metabolic activity. 

On the metabotropic side, we have not just two but 
at least eight receptor molecules. We will not cover 
them in detail. However, it is useful to remember why 
so many different receptors exist. One reasonable story 
is that glutamate is involved with 90% of the cortical 
(excitatory) synapses. But those synapses have many 
different functions, ranging from visual perception to 
executive functions and the processing of emotional 
conflicts. The situation is comparable to computers on 
the World Wide Web. In one sense, every computer on 
the Web makes use of the same transmission code (like 
the ASCII set of letters and numbers). But computers 
also need to be uniquely identifiable, so that an e-mail 
message can be sent to the right recipient, who can¬ 
not be confused with somebody else. Different recep¬ 
tor types may therefore have the function of uniquely 
identifying a particular synapse, or type of synapse, 
perhaps in a particular region of the brain, or subserv¬ 
ing a distinctive cognitive function. If the brain makes 
use of very widespread neurochemicals, it must also 
have ways of distinguishing among the synapses that 
use that particular signaling molecule. (In the nature 
of biology, over hundreds of millions of years of evo¬ 
lution the brain have exploited both of those meth¬ 
ods of differentiating synapses and their functions. 
But glutamate and GABA are good examples of very 
widespread neurotransmitters that inevitably require 
multiple receptor systems.) 


When enough postsynaptic receptors are triggered 
they can depolarize the postsynaptic cell, either iono- 
tropically by allowing the free flow of ions across its 
membrane or by triggering deeper molecular mecha¬ 
nisms. Given enough depolarizing molecules coverg¬ 
ing in a brief period of time, usually from multiple 
neurons all impinging on the postsynaptic cell, the new 
cell can set off a spike in its own axon. Or it may merely 
experience a voltage change in its input branches, the 
dendrites. Recent evidence and theory suggest that the 
graded potentials of the dendrite network may have 
their own information-processing functions. However, 
we will focus mainly on the axonal action potential as 
the primary signaling function of the neuron. 

The glutamate receptor has a long evolutionary his¬ 
tory, as you might guess from its close relationship 
with basic biological molecules like glucose and the 
essential amino acid glutamine. Japanese, Chinese, 
and French cuisine specialists have long identified the 
taste component variously called 'freshness/ 'tasty/ 
'meaty/ or 'brothy/ which is part of the good subjec¬ 
tive taste of protein foods. That taste is now called 
umami, and has been found to be one of the five bio¬ 
logically innate taste receptors located on the human 
tongue. Biologically, it seems likely that umami cor¬ 
responds to freshness in flesh foods, like meat or fish. 
Since mammals must be able to distinguish between 
fresh and spoiled foods, an umami receptor may 
confer a significant survival advantage. Chemically, 
umami involves (among others) the mGlu recep¬ 
tor number 4 (mGlu4), shown in Figure 16.31. Figure 
16.32 shows how it looks in detail as a membrane 
receptor located on the tongue. This is another exam¬ 
ple of the great generality of biologically fundamental 
molecules, the fact that they are used over and over 
again for good reasons in many different species, and 
for many different functions. 

Glutamate is a key to a number of brain-related dis¬ 
orders, including depression, anxiety (especially in its 
relationship to GABA), and the damage that occurs 
after a stroke (excitotoxicity). Glutamate synapses are 
believed to utilize as much as 90% of the energy sup¬ 
ply of the brain. They are crucial for learning, and as 
the primary excitatory neurotransmitter of the cortex, 
for other cortical functions as well. 


5*8 Ion channels that are opened by other 
molecules 

A ligand is a molecule that ties into a receptor (from 
Latin, ligare, 'to tie/ and words like ligament or 
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FIGURE 16.32 Umami: the taste of glutamate. Umami is the 
taste of freshness, according to Japanese, Chinese, and French cui¬ 
sine experts. It was first isolated from soy sauce in Japan in 1907, 
and is also an important taste ingredient in many fish and meat 
broths and sauces. Chemically, umami includes the mGluR4 recep¬ 
tor for glutamate (one of the metabotropic glutamate receptors that 
is used throughout the brain as well as on the tongue). Other Glu 
receptors are also involved. Notice that the receptor triggers a G 
protein that links to cAMP (cyclic AMP), one of the basic energy 
molecules of the cell. Source : Squire et at ., 2008. 


ligature in medicine). Glutamine is one of the neu¬ 
rotransmitters that opens ion channels in the target 
membrane, thereby allowing ions to flow freely, push¬ 
ing the membrane voltage toward depolarizing. As 
you can see from Figure 16.33, there are a number of 
other molecules that can open up ion channels, includ¬ 
ing acetylcholine, GABA, serotonin (5HT), and gly¬ 
cogen. Each image in Figure 16.33 shows a different 
ligand-gated ion channel, seen from an imaginary slice 
through the membrane, and from the outside of the 
cell. Thus the green image represents a specific 'Cys- 
loop' receptor for the acetylcholine, serotonin, GABA, 
and glycogen. (R represents Receptor.) As you can see, 
the second, orange image shows the side view of the 
ionotropic glutamate receptor, including NMDA and 
AMPA, but also kainate (which was discovered more 
recently). ATP, the energy-carrying molecule, also acts 
as a ligand to open ion channels. 

G-coupled pathways are extraordinarily important. 
About half of medical drugs are designed to target these 
pathways. G-protein receptors are sensitive to light, 
odors, hormones, and neurotransmitters (Figure 16.34). 

Given trillions of events per second in the brain, 
it may seem odd that the cortex has only two major 



FIGURE 16.33 Ligand-gated ion channels. Source: Gopala- 
krishnan and Briggs, 2007. 


neurotransmitters, glutamate and GABA. In fact 
more than 100 neurotransmitters are known, and no 
doubt some remain to be discovered. But the coding 
possibilities of neuronal information processing is 
further enriched with a multiplicity of different recep¬ 
tors for each molecular messenger (see Figure 16.35 
for an example of a neuroimaging study looking at 
dopamine receptors in human and monkey brains). 
Thus the same neurotransmitter can have quite dif¬ 
ferent effects in different synapses, depending upon 
which receptor it activates. 

Reutilization is one of the great themes in biology. 
All land-dwelling animals have lungs to enable 0 2 - 
C0 2 exchange. Reptiles like crocodiles can use their 
lungs as air bellows, to emit roars. Mammals make a 
range of sounds by forcing air from the lungs to set 
the vocal cords in vibratory motion, and humans have 
evolved ways to shape the flow of vibrating air to make 
vowels and consonants. Vocal speech and language are 
therefore entirely dependent on the prior adaptation of 
lungs, for biological reasons that have little to do with 
speech and language. Reutilization of a prior adapta¬ 
tion occurs at every step in that story. 
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FIGURE 16.34 Metabolic steps triggered by receptors. Above, transmitter molecules dock on a receptor 
on the left (A), and on a metabotropic G-protein coupled receptor on the right (B). G-proteins are preexisting 
molecules belonging to the energy cycle of every cell, the cyclic-AMP cycle. As so often happens, the brain 
reutilizes that basic biochemical process of all living things for the purpose of neurotransmission. The lower 
level of this figure presents an expanded version of the upper level. Notice that proteins are long string-like 
chains of molecules, threaded in and out of the double layered lipid membrane. On the lower left, the NH 2 
and COOH fractions exist in all amino acids, the constituents of all proteins. Source: Squire et al., 2003. 



FIGURE 16.35 Distribution of dopamine D2/D3 receptors in the brain. By designing radioactive molecules 
with chemical affinities to dopamine receptors (in this case D2 and D3), it is possible to record PET scans of the entire 
brain. On the left, the D2/D3 receptor distribution of the human brain, with the rhesus macaque in the right. Notice 
the very high levels in the thalamus (Th) in both humans and other primates. Source: Christian et ah, 2009, Figure 6. 


The neurochemicals we discuss in this chapter are 
pretty much all reutilized for brain functions from an 
earlier and wider biological role. Glutamate is derived 
from the essential amino acid glutamine, which in 


turn is made from glucose, a basic fuel of the cell. 
Metabotropic receptors reutilize the prior metabolic 
machinery of each cell. This is an important gener¬ 
alization, because it also allows us to understand and 
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FIGURE 16.36 The energy molecule ATP is also a neurotransmitter. Source : Verkhratsky and Krishtal in Squire, 2009. 


make it easier to remember the dizzying dance of mol¬ 
ecules in the brain. One great source of simplification 
is the fact of reutilization of basic life molecules for the 
purposes of running the brain. 

Figure 16.36 shows another example, the case of the 
energy molecule adenosine triphosphate (ATP), which 
is utilized in every cell in the body. ATP also functions 
as a neurotransmitter, sometimes in collaboration with 
glutamate or GABA, and always with the aid of glial 
cells, in this case the astrocyte. 


5*9 Cleaning up the synapse: deactivation, 
diffusion, recycling 

Regulating the concentration of bodily chemicals is 
crucial. We depend upon the right amounts of the 
right molecules for survival, and can develop dis¬ 
ease due to overproduction or underproduction. All 
chemical reactions in the body produce some free rad¬ 
icals, fractions of molecules that carry an extra plus or 
minus charge. Among the most common are reactive 
oxygen species such as H 2 0 2 and OH-, but also pro¬ 
tein fractions like the amine fraction of amino acids. 
Because they are charged particles these charged mol¬ 
ecules can do a great deal of damage, since they can 
interact with any other particle with opposite charge. 
Their action is sometimes called 'cross-linking' - the 
cause of wrinkling of the skin by uncontrolled mak¬ 
ing of random links. Such constant, random reactions 
are a major factor in aging and age-related diseases 


like atherosclerosis. To defend against free radicals 
the body has a large collection of antioxidants that can 
'mop up' reactive species, including vitamins and cer¬ 
tain oils. 

Similarly, the physiological response to injury 
is inflammation, a complex cascade of cellular and 
molecular events that is vital for healing. However, 
chronic inflammation is also a major factor in degen¬ 
erative diseases, including neurodegenerative disor¬ 
ders like Parkinson's and Alzheimer's disease. Again, 
the regulation of chronic inflammation is essential for 
maintaining a healthy nervous system. 

The brain is the most energy-intensive organ in the 
body, certainly under modern living conditions, when 
most human beings are not walking or running dur¬ 
ing the day. The brain therefore generates a large con¬ 
centration of free radicals and potential inflammatory 
events. The synapse and the extracellular space are 
two locations that are vulnerable to a loss of chemi¬ 
cal control because they are not embedded in the self¬ 
regulation machinery of the cell (which develops its 
own degenerative events over time, however). 

Cleaning up the synapse is a good illustration of the 
general need to control the buildup of toxic molecules. 
We have already mentioned that glutamate, the most 
common neurotransmitter in the brain, becomes toxic 
if it is not removed from the synapse quite rapidly. 
Excitotoxicity (glutamate toxicity) is the most common 
cause of brain damage after a stroke, a surprising find¬ 
ing. A stroke involves the rupture of a blood vessel in 
the brain. But the primary effects of blood seeping 
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FIGURE 16.37 Transporters recycle dopamine back to the source. Axial SPECT images 
of a radioactively tagged dopamine transporter molecule. The caudate nucleus and putamen 
(basal ganglia) are labeled. The left image is an anatomical slice of a human brain post-mor¬ 
tem, to illustrate the level of the section. The middle image shows a normal brain, with high 
levels of the radioactive signal in the caudate and putamen. On the right is the corresponding 
image in a patient with Parkinson's disease, which is caused by a loss of dopamine neurons 
in the substantia nigra (SN), which project their dopaminergic axons over wide regions of the 
forebrain. The basal ganglia are modulated by SN neurons involved with motor control, one 
of the major deficits of Parkinson's disease. Source : Newberg and Alavi, 2009. 


outside of the circulatory system, and perhaps tearing 
and disrupting other tissues, are less harmful than the 
subsequent damage due to free radicals, inflamma¬ 
tion, and excitotoxicity. The treatment for stroke has 
therefore shifted toward treating the secondary effects 
of stroke, which begin immediately after a blood ves¬ 
sel ruptures. 

The nervous system has evolved a number of ways 
to keep the concentration of potentially harmful mol¬ 
ecules in the synapse under precise control. The three 
general types of synaptic 'clean-up' are: 

1 Diffusion of the neurotransmitter molecule, which is 
typically the case for the gas-state neurotransmitter 
nitric oxide (NO). NO was not known to be a neu¬ 
rotransmitter until the past decade or so, but has 
since turned out to play a major role at many syn¬ 
apses, including perhaps the most famous exam¬ 
ple, its use in Viagra. However, NO has many other 
functions in the nervous system. Because it is a gas, 
and is secreted rapidly in very small quantities, it 
diffuses safely out of the synapse very quickly. 

2 Degradation. Acetylcholine is degraded by the 
enzyme acetylcholinesterase (-ase is the general 
suffix for deactivating enzymes). Enzymatic deg¬ 
radation may occur by literally cutting the neuro¬ 
transmitter into smaller pieces, something that is 
often possible with long protein molecules. The 
components can then be recycled into the presyn- 
aptic neuron. 

3 Recycling or 'transport' of the neurotransmitter 
back to the source neuron. Dopamine is perhaps 
the best-known example of a recycling neurotrans¬ 
mitter. Figure 16.37 shows a SPECT scan using 


a radioactively tagged molecule attached to the 
dopamine transporter molecule. The brain on the 
left is an anatomical slice post-mortem; the mid¬ 
dle photo is a SPECT image of a normal subject, 
showing high concentration of the dopamine 
transporter, especially in the basal ganglia and tha¬ 
lamus (caudate nucleus and putamen); and on the 
right side, a SPECT scan of a Parkinson's patient 
with impaired dopamine production. The last 
image has lower levels of dopamine transporter, 
probably reflecting the lower levels of dopamine 
in the patient's brain. However, it is possible that 
Parkinson's disease disrupts the dopamine trans¬ 
porter in some more direct way. 

6.0 NEUROMODULATORS 

Neuromodulation differs from neurotransmission 
in that signaling molecules are very widely distrib¬ 
uted through parts of the brain. The same molecule 
may play two different roles, depending on where it 
is secreted. Acetylcholine and glutamate can either be 
neurotransmitters or neuromodulators. As neuromod¬ 
ulators they modify large swaths of brain territory by 
regulating local synaptic signaling among billions of 
neurons and synapses. For example, neuromodulators 
may change the baseline membrane voltage of a pop¬ 
ulation of cells, and thereby modify their local synap¬ 
tic signaling. By switching neuromodulators between 
waking, sleep, and REM dreaming, the brain can con¬ 
trol the balance between sensory input versus inter¬ 
nal processes, for example. In waking, sensory signals 
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flow freely, but they may be blocked or turned down 
during slow-wave sleep and REM. The difference is 
controlled by the widespread neuromodulation of 
those states. 

Neurotransmission is usually faster than neuro¬ 
modulation, and is more likely to be ionotropic, while 
neuromodulation is slower and more likely to be 
metabotropic. Notice also that multiple neuromodu¬ 
lators might be switched on at the same time. The 
waking state, for example, is Turned on' when ace¬ 
tylcholine, serotonin, norepinephrine, and dopamine, 
among others, simultaneously begin to be distributed 
in overlapping regions of the cortex and thalamus 
(see Chapter 8). Table 16.2 shows why the brain needs 
to have multiple capacities enabled at the same time 
during the waking state, capacity being enabled by 
a different neuromodulator: such as alertness along 
with motivation, plus sensory clarity (high signal-to- 
noise ratio), plus accurate planning and motor control. 
During waking we want to use all of those interacting 
capacities as needed. 

Neuromodulating nerve cells typically are located in 
small clumps near the bottom of the brain - in the brain¬ 
stem or the basal forebrain - where they 'spray' their 
messenger molecules over large tracts of the forebrain 
(Figure 16.39). The conscious waking state is turned on 


TABLE 16.2 Major neuromodulator functions 


N euromodulator 

Some major functions 

Norepinephrine 

Maintains high signal-to-noise ratio in 
sensory systems 

Serotonin 

Protects from information overload and 
reduces cross-talk between different 
sensory channels 

Biogenic amines and 
acetylcholine: 

Dopamine 

Maintains psychomotor and 
motivational focus and alertness 
(conscious arousal) 

Acetylcholine 

Mediates attention and conscious 
arousal (waking and alertness) in all 
sensory systems 

Beta endorphin 

Counteracts homeostatic imbalances; 
enables pleasurable feelings 

Corticotropin releasing 

Promotes the effects of stress and 

factor (CRF) 

negative emotional stimuli 

Neuropeptides 

Vasopressin/oxytocin 

Vasopressin promotes male-type 
persistence; oxytocin promotes 
female-type nurturance and 
acceptance 

Cholecystokinin 

Regulates emotional system, feeling, sex, 
exploration, and pain 


Source: Northoff and Panksepp, 2008. 


by neuromodulators such as acetylcholine, norepine¬ 
phrine, and dopamine. Together, they enable a huge 
repertoire of different cognitive tasks. Each of those 
tasks involves a combination of local neurotransmitters 
like glutamine and GABA, along with neuromodulators 
that control the overall brain state that is needed for a 
task like perception or working memory to take place. 

Thus the substantia nigra (SN, literally the 'dark sub¬ 
stance' in Latin) is a small wedge of nerve cells located 
just above the brainstem, which send their axons 
spreading upward to modulate the activity of billions of 
other neurons (Figure 16.38). The SN has about 10 000 
dopaminergic neurons, not many compared to the tens 
of billions in the cortex. But their influence is huge - 
and correspondingly, when dopaminergic SN neurons 
are lost, as in Parkinson's disease, the loss can be very 
serious. Dopamine modulation is involved with basic 
functions like pleasure and reward seeking, sleep and 
waking, nicotine and stimulant addiction, working 
memory, voluntary motor control, eye movements, and 
goal-directed learning. It is a huge set of functions. (Like 
other brain structures all neuromodulating nuclei come 
in pairs, one on each side of the brain. Each SN has two 
distinct regions, called pars compacta for the 'dense 
part,' and 'pars reticulata' for the 'network-like part.') 

Most synapses combine local transmission with 
regional neuromodulation. For example, a visual neu¬ 
ron in cortex might behave quite differently depending 
on global brain states, like sleep or waking, which are 
controlled by brainstem nuclei that project their axons 



FIGURE 16.38 Neuromodulation: basal brain nuclei shape 
widespread brain functions. Neuromodulation from small nuclei in 
the basal brain shapes the general activity of the cortex and basal 
ganglia. Source: National Institute of General Medical Sciences, 2009. 
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to cortex, distributing neuromodulator molecules like 
acetylcholine and dopamine far from their cells of 
origin. 

7*0 LEARNING 

Chapter 9 covered learning and memory at the large 
scale of the brain. Here we will explore some of the 
molecular aspects, which have been studied very 
intensively in the past few decades. Scientific progress 
in this field is likely to have great impact, not just in 
coping with learning disorders, but in greatly improv¬ 
ing learning over the lifetime. 

All brain regions are involved in learning, memory, 
and plasticity, which can be considered as different 
methods for evoking long-lasting adaptive changes in 
the brain. The most typical example of such adapta¬ 
tion is believed to be Hebbian learning, defined as the 
growth of synaptic connectivity due to coordinated 
neuronal firing. The slogan for Hebbian learning is 
that 'neurons that fire together, wire together.' That 
is a good first approximation to the neuronal level 
of learning. The major point here, however, is that 
this fundamental kind of learning might be observed 


almost anywhere in the brain where two neurons join 
at a synapse. Figure 16.40 shows the wide range of 
learning processes that are known to occur, along with 
the brain regions that support them. There is no sin¬ 
gle learning region of the brain. All brain functions can 
adapt to changing conditions. The sensory (posterior) 
half of the cortex adapts to changing sensory condi¬ 
tions, most famously in early visual learning in kittens 
(Hubei and Wiesel, 1968). An everyday example is our 
growing experience of a new variety of music or art, 
where simply paying attention to the song allows for 
a deeper and often more satisfying experience of the 
same physical event. 

The frontal half of cortex supports the learning of 
executive functions (Chapter 12), working memory, 
speech and language-related learning, and of course 
motor control. Merzenich and coworkers (1984) 
showed that the loss of a single finger will cause an 
adaptive reorganization of the hand and finger maps 
in cortex. Whereas previously all five fingers were 
well represented (as shown by stimulation of the 
fingers, with corresponding activity recorded from 
the cortical finger maps), after amputation the corti¬ 
cal map lowered its responding to the missing finger 
and increased the representation of the remaining 



FIGURE 16.39 Five major neuromodulators. Notice that all of these neuromodulators 
project axons from small nuclei below the cortex, spreading their neurochemicals to wide regions, 
both cortical and subcortical. Only a few thousand cell bodies in these nuclei therefore have mas¬ 
sive effects in the rest of the brain, controlling sleep and waking, pleasure and pain, alertness and 
working memory. When neuromodulating cells die off, as in the case of dopaminergic cells in 
Parkinson's disease, the results can be debilitating and ultimately deadly. Source : Adapted from 
Gu, 2002, Figure 1. 
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FIGURE 16.40 Learning occurs everywhere in the brain. 
Memory storage occurs in the same regions that are used in active 
tasks. Thus perceptual memory appears to involve selective 
strengthening and tuning of synapses and receptive fields in the 
perceptual half of cortex. The amygdala, hippocampus, and medial 
prefrontal lobe are involved in different aspects of emotional learn¬ 
ing. Declarative memory (i.e. memory for conscious episodes and 
concepts) is mediated by the hippocampal complex, but after con¬ 
solidation, declarative memories are also stored in the neocortex 
(see Chapter 12). Thus learning of one kind or another occurs in all 
parts of the brain. Source: Eichenbaum, in Squire, 2003. 


neighboring fingers. Thus cortical representation was 
optimized for the working fingers. This is a general 
principle that applies to all sensory and motor regions 
of cortex, and probably to prefrontal regions as well 
(Merzenich et al., 1984). 

A similar reduction of cortical finger map size occurs 
when we use one finger repetitively. When the same 
finger movement is repeated over and over again, the 
cortical representation of that finger will decrease, as 
if the cortex is treating it as a redundant (highly pre¬ 
dictable) event. However, subcortical innervation of 
the finger does not disappear, because the brain is still 
controlling and monitoring finger movements. 


7.1 The hippocampal complex 

Declarative memory for conscious events seems to pri¬ 
marily occur via the medial temporal lobe, what we have 
called the hippocampal complex (because it is broader 
than just the hippocampus by itself). Chapter 8, Figure 
8.16, shows a dynamic circuit for the temporary storage 
of conscious episodes during the waking state, including 



FIGURE 16.41 A hippocampal slice culture. This slice of hip¬ 
pocampus can be kept functioning in culture, allowing experiment¬ 
ers to apply different neurotransmitters and record from different 
kinds of neurons at the same time. While the typical hippocampal 
slice is rat tissue, it has striking similarities to human hippocampus. 
The figure shows three types of hippocampal neurons, CA1, CA3 
(named after the Cornis Ammonis, the curved section of hippocam¬ 
pus that looks like a horn), and Schaffer collaterals. Source: Stevens 
and Sullivan, 1998. 


the sensory cortex, thalamus, and the hippocampal com¬ 
plex. Keep in mind that a great deal of episodic learn¬ 
ing also engages implicit circuits, so that conscious cues 
normally carry their own implicit (unconscious) seque¬ 
lae, which must also be encoded in the brain. A well- 
studied example is the role of word sequence learning 
in children, and the implicit induction of grammatical 
rules and regularities as a result of explicit learning. A 
three-year-old child may say 'Mommy, airplane!' but the 
implicit expression is richer than those words. It implies, 
'Mommy, pay attention to me, and look there, where I'm 
pointing, where there is an airplane going over in the 
sky.' That implicit knowledge must also be learned, both 
by mother and child. There is likely to be no such thing 
as isolated explicit learning; all explicit cognition also has 
an implicit side to it. 

Episodic learning is well established to take place 
in at least two stages, the first one lasting on the order 
of seconds, minutes, and hours, and involving the 
hippocampal region (MTL), which receives input 
from vision and the other senses (see Figure 16.41 for 
a typical hippocampal slide preparation). In the case 
of vision, for example, the temporal lobe involves 
the integration of different visual feature streams 
into coherent objects and events. It is those conscious 
objects and events that become the contents of immedi¬ 
ate memory. See Figure 16.42 for a cross sectional slice 
of the hippocampus showing the great complexity and 
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FIGURE 16.42 Endocannabinoid activity in the hippocam¬ 
pus. Endogenous cannabinoid molecules are important in learning 
and synaptic plasticity, (a) This photograph of a cross-section of the 
hippocampal formation uses in situ hybridization to color stain the 
different tissues. The brown colors indicate CB1 receptor mRNA. 
Cell nuclei are stained blue. GABA-ergic neurons are shown by the 
black arrow and glutamatergic pyramidal cells with the white 
arrow, (b) shows the co-expression of CB1 receptor mRNA (red) 
with a GABA marker (GAD), (c) shows the co-expression of the CB1 
receptor mRNA (red) with a molecular marker of glutamate trans¬ 
porter type 1 (VGLUT1). Notice the great complexity and richness 
of the molecular coding involved in hippocampal learning. Source : 
Lutz and Marsicano, in Squire et al., 2009. 

richness of the molecular coding involved in hippoc¬ 
ampal learning. Those memories are labile (unstable), 
so that they are vulnerable to interference. An electri¬ 
cal shock delivered soon after learning will disrupt 
new memories but not old and stable memories. 

The traditional question is what molecular and cel¬ 
lular events cause long-term memories to be estab¬ 
lished. However, it now appears that even immediate 
memory, the dynamic circuit shown in Figure 8.11, 
also involves molecular and epigenetic activities. We 
will explore this point later (Wang, 2003). 

7.2 Glutamate, GABA, LTP, and LTD 

If learning involves making connections between neu¬ 
rons, conceptually we can have two types: either an 


increase or decrease in the event probability of the link. 
Traditionally this has been called long-term potentia¬ 
tion (for strengthened links) and long-term depression 
(for weakened links). They correspond conceptually 
to excitation and inhibition, although the electrochemi¬ 
cal mechanisms are not identical. In fact, over decades 
of efforts to identify the brain basis of LTP and LTD it 
has become clear that there are many mechanisms of 
LTP and LTD in the brain, just as there are many vari¬ 
eties of neurons, synapses, neurochemicals, and so on. 
Nature thrives on a diversity of means, using many dif¬ 
ferent ways to accomplish the same fundamental func¬ 
tions. One reason for the vast diversity of pathways and 
mechanisms at all levels of biological organization is 
that functional redundancy leads to stability in the face 
of changing conditions (Edelman & Gaily, 1992). Human 
engineers also use multiple backup systems in complex 
projects to ensure continuity of functioning if something 
goes wrong. 

Figure 16.43 shows how LTP and LTD can be stud¬ 
ied in hippocampal slices, and in many neural prepa¬ 
rations. The stimulus in this case is labeled Tetanus/ 
a small electrical shock. The vertical axis shows mil¬ 
livolts per milliseconds, as an index of neuronal mem¬ 
brane voltage over time. That level roughly doubles 
immediately after the tetanic shock and continues at a 
high level for more than 90 minutes. It is the duration 
of the higher activity level that suggests that we are 
not dealing with excitatory or inhibitory neural trans¬ 
mission but with a longer-term change in the neuronal 
circuit. 


73 Glutamate synapses as early models of 
synaptic learning 

The first LTP system to be studied in detail was the 
glutamate synapse, by far the most common excita¬ 
tory synapse in the brain. 

Figure 16.44 shows an example of Pavlovian condi¬ 
tioning in one of the best-studied organisms in biology, 
the fruit fly. Drosophila melanogaster. Drosophila is quite 
a smart creature, with as many as 100 000 neurons 
in its brain. It is a skilled flyer, pathfinder, pattern- 
recognizer, learner, and social creature - with male- 
male aggression in competition for a female or food, 
and sexual selection for the most suitable partner for 
reproduction. Olfaction is one of its developed senses. 
Odors like the scent of fruit sugars can guide the fly 
to food, warn it away from toxins, and serve as phero¬ 
mones to attract it to the opposite sex. The fruit fly can 
learn to associate an arbitrary odor with an aversive 
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B1. Control B2. 90 min. B3. Superimposed 



FIGURE 16.43 Measuring long-term potentiation (LTP) and long-term depres¬ 
sion (LTD). Notice that this figure shows only the input (tetanus shock) and output 
(millivolts per milliseconds) of the preparation. In the case of a hippocampal slice cul¬ 
ture we may be seeing the activity of a single synapse or of entire connected layers of 
neurons (like CA1, CA3, and the Schaffer collaterals). Source : Byrne, in Squire et ah, 
2003. 


event, like a shock, because electrical tetanus (shock) 
triggers signaling in all neural tissues. Neurons com¬ 
municate electrochemically and are therefore sensitive 
to electrical pulses. Shock can also be painful and 
aversive, activating neural circuits for harm detection 
and avoidance. 

Figure 16.44(a) shows the experimental apparatus, 
a small T-maze for the fly, baited with two end boxes 
with two different odors. Odor 1 serves as the condi¬ 
tioned stimulus, the arbitrary signal for the uncondi¬ 
tioned stimulus, the electrical shock to the fly's legs as 
it is walking through the horizontal plastic tube. When 
Odor 1 is followed by shock to its legs, the fly quickly 
learns to go in the direction of the safe odor. Odor 2, 
and stay away from Odor 1 at the choice point in the 
T-maze. A stream of air helps to move the fly along, 
and keeps it from going back to earlier points in the 
maze. 

Figure 16.44(b) illustrates how associative learn¬ 
ing is believed to occur at glutamate synapses in 
Drosophila. The theoretical question is simple, and 
straight from Donald Hebb: If the two neurons fire 
together, will they wire together? Will the chemi¬ 
cal synapse be able to detect events that go together, 
like Odor 1 and the electrical shock? And can the syn¬ 
apse be strengthened and made more likely to go into 


action the next time Odor 1 is encountered? The ques¬ 
tion may seem simple, but the molecular machinery 
needed to do the job is not trivial (see Figures 16.45, 
16.46, and 16.47 for schematic illustrations of how this 
occurs). What's more, there is good reason to think 
that the glutamate synapse that supports learning in 
D. melanogaster is highly conserved among other spe¬ 
cies, including human beings. 

One interesting way to strengthen synaptic links 
between two neurons is to literally tie the two cells 
together, using cell adhesion molecules. Figure 16.48 
shows how this works in concept. 

Learning always requires structural changes in 
neurons - growing dendritic spines, making new syn¬ 
apses, and building up the metabolic machinery to 
sustain higher rates of neuronal activity. A great deal of 
learning is believed to involve the Hebbian growth of 
connections between neurons, following the rule that 
'neurons that fire together, wire together' (Chapters 3 
and 10). Since 'wiring together' requires growing big¬ 
ger and more interconnected neurons, epigenetic con¬ 
trol of protein expression becomes an essential part of 
the story. By definition, learning involves 'experience- 
dependent changes in the brain.' The experiences we 
have during our waking hours constitute environmen¬ 
tal input, and the resulting brain changes require new 
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FIGURE 16.44 Pavlovian learning in the fruit fly. Classsical 
conditioning in the fruit fly allows the study of associative learn¬ 
ing in a relatively small nervous system, using fear conditioning of 
an arbitrary odor to signal an electrical shock. The two ionotrophic 
receptors for glutamate are NMDA and AMPA (the R stands for 
Receptor). They directly open ion channels in the target membrane, 
including calcium and magnesium channels. Notice the firing rate 
of the two neurons (marked by a 'comb' shaped set of vertical lines). 
Long-term potentiation requires additional changes deeper in the 
cells, to enable structural changes in the synapse. Source: Glanzman, 
2005. 
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FIGURE 16.45 Long-term potentiation using NMDA and AMPA receptors. A closer look at the firing-rate depend- 
ency of the NMDA receptor for LTP On the left, in the absence of glutamate transmission across the synapse, NMDAR is 
blocked by magnesium ions (small black dot). The center figure shows the same synapse at the basal firing rate of the 
presynaptic cell, the background rate that does not signal any particular event. Glutamate (red) diffuses across the syn¬ 
aptic gap, but the NMDA receptor is still blocked by the magnesium ion. On the right, high-frequency signaling from the 
presynaptic cell triggers the NMDA receptor. Thus the synapse operates as a high-bandpass switch that opens only 
when provided with a high-frequency input. Source: Doherty, Fitzjohn, and Collingridge, in Squire et al., 2009. 
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FIGURE 16.46 Long-term potentiation: a closer look at NMDA. Learning is not limited to single 
neurons. Rather, it is usually a network event, involving multiple excitatory and inhibitory synapses. In 
this case, on the top left, an excitatory (glutamatergic) cell innervates an inhibitory cell (GABA-ergic), 
which in turn lowers the firing probability of the target cell at the bottom of the figure. Thus the target 
cell receives both inhibitory and excitatory inputs. On the right side, the membrane voltages due to 
GABA and glutamate are added, with the AMPA receptor yielding a positive spike and the NMDA recep¬ 
tor being blocked by a magnesium ion (black dot). Two GABA receptors are represented, GABA (A) and 
GABA (B), with slightly different effects on the target cell. The EPSP (excitatory postsynaptic potential) is 
the "bottom line' of the system, which yields a spike followed by a refractory period. Circuits like this can 
become quite complex and adaptable. Source : Doherty, Fitzjohn, and Collingridge, in Squire et al., 2009. 
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FIGURE 16.47 Translational and transcriptional activation for LTP/LTD. LTP and LTD require pro¬ 
tein synthesis to grow the number of synapses, add synaptic spines, and increase the flow of neurochemi¬ 
cals (typically proteins) to all parts of the system. When a synapse is strongly stimulated both translational 
(mRNA) and transcriptional (DNA-mRNA) mechanisms are activated. 
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FIGURE 16.48 Cell adhesion molecules stabilize the synapse. The red cadherin proteins work to 
keep the two neurons at the synapse together, thereby facilitating neural signaling across the synapse. 
Cell adhesion molecules have numerous functions, and seem to be essential for learning and plasticity. 
Source : Tai, Kim, and Schuman, 2008. 


bricks and mortar to permit active neuronal networks 
to adapt to those daily encounters with the environ¬ 
ment. Like an automobile factory, each cell responds 
to adaptive demands by making more of a variety of 
different products (see Figure 16.49). 

We have described learning in Hebbian terms, as the 
strengthening of synaptic links among neurons. That 
picture is correct as far as it goes, but it is only part of 
the adaptive apparatus of the living brain. It is believed 
that there may be non-Hebbian learning mechanisms 
as well. Adaptation to the world, to ourselves, and to 
each other, is after all what the brain does. Given hun¬ 
dreds of millions of years of evolution, brains have 
evolved many mechanisms of adaptation; they all 


involve Teaming' in a broad sense of that word. The 
terms 'learning/ 'memory,' and 'neural plasticity' have 
now become near synonyms. Historically, the term 
learning was influenced by the associative learning 
tradition, particularly for Pavlovian and operant con¬ 
ditioning. Plasticity was first studied in the realm of 
perceptual learning, such as Hubei and Wiesel's semi¬ 
nal work on the adaptation of occular dominance col¬ 
umns in kittens that were deprived of input to one eye 
or the other. Plasticity was studied in terms of imprint¬ 
ing during critical periods , the rapid learning that takes 
place in a restricted period of time in young mammals, 
and perhaps in the fetus even before birth. Imprinting 
during critical periods is key to the early growth of 
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FIGURE 16.49 Some ways for synaptic signaling to be strengthened. There are 
multiple ways to increase synaptic strength. In the case of serotonin (5-HT) this table 
suggests four different ways. One is a change in the structural supports of the cell, the 
cytoskeleton, including an increased number of synapses. Serotonin can also evoke a 
synaptic tag, triggering gene expression to support greater synaptic stabilization. 
(Synapses, like other cellular elements, are constantly cycling and having to be replen¬ 
ished.) The neurotransmitter can also make morphological changes in pre-existing syn¬ 
apses, enlarging them, or adding to the flow of molecules diffused across the synaptic 
cleft. And finally, serotonin can work via a second messenger like a G-protein, enhance 
translational productions (involving mRNAs), or new gene expression (by epigenetic 
changes in DNA expression), leading to a cascade resulting in greater transmitter 
release. Source : Sossin, 2008. 


the visual brain, for example, to visuomotor learning, 
speech perception and production, and much more. 

But the term plasticity has greatly broadened in 
meaning. Adults are now known to have cortical 
plasticity throughout the lifetime, even if it is less dra¬ 
matic than the early plasticity of the visual brain. Major 
changes in the sensory or motor domain, such as the 
loss of a finger, trigger immediate plasticity in the allo¬ 
cation of neurons in the sensorimotor cortex. But even 
simply practicing a repetitive movement, as in typ¬ 
ing the same keystroke, will also trigger reassignment 
of the cortical map for that finger, and for the neigh¬ 
boring fingers. Thus 'plasticity' is merging with other 
labels for the adaptive function of the nervous system. 
Sometimes it is useful to emphasize the differences, but 
we will treat them as parts of the same topic. 

Memory, as a technical term, has its own scien¬ 
tific history, with an emphasis on the nature of stored 
representation. But there is a tendency to use those 
words interchangeably when it comes to the brain basis. 
One reason is that it is hard - perhaps impossible - to 
find truly different brain events corresponding to the 
traditionally separate terms. 

The brain begins learning as soon as it is placed in 
any novel environment - which is any environment 
at all. New stimuli tend to trigger a massive orient¬ 
ing response, including widespread brain activation. 


When the animal feels safe enough to explore the 
novel environment, it will begin to gather informa¬ 
tion by sniffing and looking. Simple novelty is enough 
to trigger attention and learning, including signifi¬ 
cant evoked potentials that sweep through the entire 
cortex. 

A rodent in a laboratory task is acutely sensitive to 
the experimenter's personal odors, for example, such 
as the odor of the experimenter's pet cat. To a labo¬ 
ratory rat the predator odor signals imminent threat, 
and some researchers report a tendency for rodents 
to hunker down in fright. Learning is hardly optimal 
under those conditions or, rather, the animal is learn¬ 
ing something different from what the experimenter 
thinks it is. Learning of experimental tasks usually 
requires the animal to feel safe enough to explore. 

In the real lab, as in real life, animals are doing all 
kinds of learning at the same time. When we look 
at brain events, that rich set of adaptive processes 
becomes apparent. 

7 A Epigenetics of learning 

As mentioned before, epigenetics has now become 
crucial to the study of learning. That is, learning 
engages the genetic controls for the neurons and glia 
that are engaged by the task. Neurotransmitters alter 







8.0 SUMMARY 


549 



FIGURE 16.50 Activity-dependent gene expression (CaMKs). 
Source : Nelson and Monteggia, in Squire, 2009. 

the DNA expression of their cells of origin and their 
target cells. Since DNA encodes the control plan for 
the production of molecules, including neurotransmit¬ 
ters, this makes sense for the presynaptic cell. But it is 
equally true for the postsynaptic cell, and for neigh¬ 
boring glia as well. 

Figure 16.50 shows activity-dependent gene 
expression in a postsynaptic cell. Rhythmic stimula¬ 
tion of neurons evokes specific gene expressions that 
enable learning to occur. Theta rhythms are known 
to occur with episodic learning in the hippocampus, 
and later on, as a vehicle for hippocampal-neocortical 
memory consolidation. Thus theta may have the effect 
of changing the very nature of the neurons it recruits. 
Such rhythmic stimulation also triggers epigenetic 
processes in cell populations. 

This particular mechanism makes use of very impor¬ 
tant calcium-dependent molecules called CamK (for 
calmodulin-dependent kinase), which help to induce 
plasticity by altering gene expression in the nucleus. 
As Figure 16.50 shows, calcium ions flow into the cell 
when the NMDA receptor opens the channels (along 
with voltage-gated calcium channels). The rise of cal¬ 
cium ions binds to the protein calmodulin (CaM), which 
activates proteins involved in the regulation of gene 
expression. 

Brain molecules are prime targets for drugs to 
improve mood disorders, dementia and stroke, brain 
injury, sexual functioning, coma, attention and memory, 
insomnia, daytime sleepiness, and more. Add a range of 
recreational and addictive drugs - often with negative 
or simply unknown safety profiles - hormones, heavy 
metal toxins, steroids, and the like, and we seem to be 
living in a world of brain-targeting chemicals. A newer 
category of pharmaceuticals, called nootropics (from the 
Greek words for mind, nous , and tropos, movement), 
aims to improve our ability to learn and stay alert. 


Some time in the future college students are likely to 
go beyond drinking lots of coffee before an exam to use 
safe and effective nootropics. 

7.5 Neurotrophic factors in learning 

In learning to play guitar our spinal motor neurons 
may need structural strengthening. For building bigger 
neurons with longer branches we need to recruit cel¬ 
lular growth mechanisms called neurotrophic factors 
(Figures 16.51 and 16.52). One of those is a molecule 
called BDNF (for brain-derived neurotrophic factor). 
Recent evidence shows that BDNF injected into the 
brain vesicles of rodents improves their learning, and 
that BDNF injections also increase slow-wave sleep. 
That is consistent with the evidence that SWS plays 
a role in memory consolidation, enabling structural 
changes for long-term memory (Chapters 8 and 9). 

8.0 SUMMARY 

In this chapter, we have discussed how basic life 
molecules - such as glucose, choline, and calcium - 
are also used for neural signaling in the brain. These 
'molecules of cognition' serve to carry brain signals 
that enable us to perform our everyday tasks. Life 
uses enzyme chemistry, making constant use of molec¬ 
ular nanomachines that speed along a second-by-sec¬ 
ond flow of chemical reactions. All brain chemicals 
act enzymatically. The life molecules are highly con¬ 
served, by which we mean that they are found across 
many species. Importantly, these molecules may per¬ 
form differing functions depending on whether they 
are acting in the body or the brain. 

All human cells have a complete copy of our spe¬ 
cies genome in their nuclei, in the form of a long, lin¬ 
ear DNA code, twisted into the well-known double 
helix. The genetic code is expressed in proteins that 
define, develop, regulate, and control the phenotype. 
In each cell nucleus DNA is transcribed into messen¬ 
ger RNAs, which then are translated into proteins. The 
Central Dogma of molecular biology claims that that 
molecular causality flows from genes to messenger RNA 
to body proteins, but not the other way around. 

Genes and molecules play different roles in dif¬ 
ferent stages of development: they shape the evolu¬ 
tion of the brain, such as the frontal lobes - including 
language, social relationships, and the ability to rec¬ 
ognize facial expressions. They control the program 
for lifelong brain development, from fetal growth to 
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FIGURE 16.51 Growth factors in learning. The body is constantly maintained by molecules called 
growth factors, such as human growth hormone. Two growth factors are also important in learning, called 
neurotrophin and BDNF (for brain-derived neurotrophic factor). Neurotrophin is driven by the firing rate of 
neurons, and by the flow of neurotransmitters across the synapse. Source : Berardi et al., 2003. 



FIGURE 16.52 Physical exercise builds brain growth factors. A ongoing balance exists between 
factors like stress and the constant rebuilding of the brain and body Exercise induces growth factors, 
helping to maintain cognitive functions, synaptic plasticity, neurogenesis, and vascular function. 
Exercise also reduces hypertension and insulin resistance. Chronic inflammation also is reduced by 
moderate exercise. It has been shown to reduce and delay the risk of neurodegenerative disorders 
like Alzheimer's and Parkinson's disease. Source : Cotman, Berchtold and Christie, 2007. 


aging. And in nerve cells, genes also control moment- 
to-moment events like learning and falling asleep. 

The environment and genes interact in all stages 
of brain development, including proliferation. 


differentiation and migration, axonal guidance, and 
synapse formation. Gene expression in human brain 
cells can be changed, within limits, by selectively turn¬ 
ing stretches of DNA on or off. Such 'epigenetic' flow 
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of control runs 'on top of' (epi-) the flow of normal 
top-down causation. Epigenetics has now become cru¬ 
cial to the study of learning. 

Messenger molecules in the brain are of two kinds: 
transmitters and modulators. We can define neuro¬ 
transmitters as having very local effects in the synapse 
between two neurons, although there might be billions 
of such neurons and synapses. To date, more than 100 
neurotransmitters have been identified. These brain 
chemicals are closely regulated: if the synapse has 
too little of a neurotransmitter, its message will not 
be sent. On the other hand, too much of the signaling 
molecule is generally toxic, and can kill nerve cells. For 
that reason there are pervasive negative feedback loops in 
any chemical pathway, much like tiny thermostats that 
keep the quantity of any molecule within safe limits. 

Glutamate and GABA are the two most common 
neurotransmitters in the brain. Ninety percent of the 
neurons in cortex use glutamate, the primary excita¬ 
tory neurotransmitter, which increases the probabil¬ 
ity of the next neuron firing when a precise number 
of molecules are released into the synapse. Too much 
glutamate is toxic and is specifically called excitotoxic. 
Excitotoxicity due to excess glutamate is one of the 
most harmful aspects of brain damage and disease. 
That is why it is essential to keep the levels of gluta¬ 
mate as low as possible, without interfering with the 
transmission of excitatory signals. 

Glutamate can evoke two molecular receptor cas¬ 
cades, one involving NMDA and the other AMPA. Both 
of those substances are vital in the process of learning. 
NMDA increases the membrane voltage, while AMPA 
decreases it. The balance of NMDA and AMPA deter¬ 
mines what kind of learning will take place at the target 
membrane of the glutamate synapse. NMDA is associ¬ 
ated with long-term potentiation (LTP), making the 
synapse more excitable over time. AMPA is associated 
with long-term depression (LTD), which makes it less 
excitable. The first LTP system to be studied in detail 
was the glutamate synapse, by far the most common 
excitatory synapse in the brain. LTP and LTD require 
protein synthesis to grow the number of synapses, add 
synaptic spines, and increase the flow of neurochemi¬ 
cals (typically proteins) to all parts of the system. 

The brain is the most energy-intensive organ in 
the body. The brain therefore generates a large con¬ 
centration of free radicals and potential inflamma¬ 
tory events. Chronic inflammation is a major factor in 
degenerative diseases, including neurodegenerative 
disorders like Parkinson's and Alzheimer's diseases. 
Regulation of chronic inflammation is essential for 
maintaining a healthy nervous system. The synapse 


and the extracellular space are two locations that are 
vulnerable to a loss of chemical control because they 
are not embedded in the self-regulation machinery of 
the cell. Cleaning up the synapse is a good illustration 
of the general need to control the buildup of toxic mol¬ 
ecules. The nervous system has evolved a number of 
ways to keep the concentration of potentially harmful 
molecules in the synapse under precise control. The 
three general types of synaptic 'clean-up' are diffusio, 
degradation, and recycling of the neurotransmitter. 

We stated above that neurotransmitters have very 
local effects in the synapse between two neurons. 
Neuromodulators , on the other hand, are produced by 
quite small clumps of cell bodies below the cortex, 
and are spread very widely. These cells - typically 
just a few thousand - project their branches through 
sizable parts of the brain. Where they terminate, they 
'spray' their messenger molecules. Thus neuromodu¬ 
lators can change large regions of the brain, whereas 
neurotransmitters act locally in nano-level synapses. 
Neuromodulation from small nuclei in the basal brain 
shapes the general activity of the cortex and basal 
ganglia. 

Remember that the same molecule may play two 
different roles, depending on where it is secreted. 
Acetylcholine and glutamate can either be neurotrans¬ 
mitters or neuromodulators. As neuromodulators they 
modify large swathes of brain territory by regulating 
local synaptic signaling among billions of neurons 
and synapses. For example, neuromodulators may 
change the baseline membrane voltage of a popula¬ 
tion of cells, and thereby modify their local synaptic 
signaling. By switching neuromodulators between 
waking, sleep, and REM dreaming, the brain can con¬ 
trol the balance between sensory inputs versus inter¬ 
nal processes, for example. In waking, sensory signals 
flow freely, but they may be blocked or turned down 
during slow-wave sleep and REM. The difference is 
controlled by the widespread neuromodulation of 
those states. 

The body is constantly maintained by molecules - 
called growth factors - such as human growth hor¬ 
mone. Two growth factors are also important in 
learning: neurotrophin and BDNF (for brain-derived 
neurotrophic factor). Neurotrophin is driven by the fir¬ 
ing rate of neurons, and by the flow of neurotransmit¬ 
ters across the synapse. Physical exercise builds brain 
growth factors. An ongoing balance exists between 
factors like stress and the constant rebuilding of the 
brain and body. Exercise induces growth factors, help¬ 
ing to maintain cognitive functions, synaptic plastic¬ 
ity, neurogenesis, and vascular function. Exercise also 
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reduces hypertension and insulin resistance. Chronic 
inflammation is also reduced by moderate exercise. 

In this chapter, we explored the molecules of cog¬ 
nition that help to carry brain signals and perform 


the computations that make the human mind possi¬ 
ble. While it is a complex topic, understanding it will 
enable us to elucidate the fundamental processes that 
underlie our mind-brain! 


9.0 CHAPTER REVIEW 

1 Name five molecules that are highly conserved. 
What were their previous functions? What are 
their brain functions? 

2 DNA operates over several time scales. What 
are they? 

3 How do Hox genes affect brains? How is the 
developmental schedule of the brain affected by 
genes? What is the role of FOXP2? 

4 What is the Central Dogma of molecular biology? 
How has it been changed by recent develop¬ 
ments? Define 'epigenetic' and give an example. 

5 Brain signaling molecules are of two kinds. 
What are they? How do their actions differ? 


6 What is the most common neurochemical in 
the cortex? The second most common one? 
What different roles do they play? 

7 How can the same signaling molecule have 
different receptors? What is a well-known 
example? 

8 How can normal neurotransmission lead to 
brain dysfunction? 

9 What two kinds of 'learning effects' are believed 
to exist at the synaptic level? What levels of the 
molecular machinery of the cell are believed to 
be involved in each of them? 

10 What two types of cells are involved in infor¬ 
mation processing in the brain? 
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TABLE 16.3 Other neurotransmitters 


Chemical 

class 

Example 

Cell of origin 

Storage 

Release and other 
mechanisms of 
initiating signaling 

Inactivation 

Receptors 

Acetylcholine 

Acetylcholine 

Neurons 

Vesicles 

Depolarization 

Enzymatic 

(acetylcholinesterase) 

Ionotropic 
(nicotinic) 
and GPCR 
(muscarinic) 

Arnino acids 

Glutamate 

Neurons 

Vesicles 

Depolarization 

Transporters 

Ionotropic and 
GPCR 

Monoamines 

Dopamine 

Neurons 

Vesicles 

Depolarization 

Transporter 

GPCRs 

Peptides 

Enkephalin 

Neurons 

Vesicles, often 
separate from 
small molecule 

neurotransmitters 

Repetitive 

depolarization 

Enzymatic (for many 
peptides relevant 
enzymes or other 
mechanisms are 
unknown) 

GPCRs p, 8, opiate 
receptors 

D-Amino 
acids 

D-Serine 

Astrocytes (glia) 

Unknown 

Activation of 
serine racemase 
followed by 
release of 
resulting d-serine 
by unknown 
mechanisms 

Transporter 

Modulatory 
(glycine) site 
on NMDA 
glutamate 
receptors 

Purines 

Adenosine 

Neurons and 
probably others 

ATP is stored in 
vesicles and 
may be rapidly 
hydrolyzed to 
adenosine after 
release; high 
basal extracellular 
levels represent 
unstored 
adenosine 

(1) Liberation 
from diverse 
extracellular 
purines by 
ectoenzymes 
(2) Facilitated 
diffusion out of 
cells 

Transporter 

GPCRs 

Gases 

NO 

Neurons 

None 

Ca2 + -dependent 
regulation of 
nitric oxide 
synthase (nNos) 
to produce NO 
for immediate 
diffusion 

Diffusion. Perhaps 
enzymatic 
degradation 

Many enzymes 
containing 
transition 
metals; such as 
soluble guanylyl 
cyclase 

Lipids 

Anadamide 

Neurons 

Unknown if stored 

Probably released 
info synapse 
but diffusion 
mechanism in 
aqueous medium 
unclear 

Transporter followed by 
hydrolysis 

GPCRs 

(cannabinoid) 


(1) These examples are not fully extendible to the class. For example, the purine adenosine triphosphate (ATP), which despite its ubiquity 
as an energy carrying molecule can also carry information [10], is colocalized in vesicles with other neurotransmitters, such as acetylcholine 
or dopamine, and is released by depolarization; while the purine adenosine is released by facilitated diffusion. ATP interacts with both 
ionotropic receptors (P2X receptors) and G-coupled receptors (P2Y receptors), while adenosine only interacts with G-coupled receptors. 

(2) As alluded to in the text, glia may, in some cases, be directly involved in neurotransmitter action: for example, by inactivation of 
neurotransmitter, such as glutamate, or by release, for example, of ATP or d-serine [11]. 
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1.0 HISTORICAL BACKGROUND 

1.1 Correlating brain and mind 

Understanding the complex functions of the brain, 
including sensation, motor function, consciousness, 
and thoughts, has always triggered our curiosity 
The brain is activated when we are engaged in solv¬ 
ing a task, such as thinking, speaking, or purposeful 
movements. The activation is accompanied by local 
changes in the cerebral blood flow in the activated, 
i.e. involved, brain regions and this forms the basis for 
functional brain mapping studies. In the following, we 
provide a brief overview of some of the milestones in 
this field. 

It was recognized early that a hemispheric lesion 
led to contralateral palsy, a loss of the ability to move 
a body part. If the lesion was on the left side, language 
disturbances could occur. In the 19th and the beginning 


of the 20th century, the functional organization of the 
brain was investigated through clinical-pathological cor¬ 
relations. Detailed descriptions of symptoms were sub¬ 
sequently (after the patient's death) related to autopsy 
findings, enabling the localization of specific functions 
in the brain. The main limitation of this method is that 
human brain lesions are accidental and thus unspecific 
in terms of localization because the damage is diffuse 
and often extends to regions without relevance for the 
neurological deficit in question. Moreover, adaptive 
brain responses may modify the neurological deficits 
over time. With the clinical-pathological correlation we 
learn about deficits that occur if a given brain region is 
injured, i.e. for which functions a given brain region is 
essential. For example, it was lesion studies that taught 
us that Wernicke's and Broca's areas are both essential 
for normal language function. 

These limitations have provided the incentive for 
parallel development of other methods that allow a 
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FIGURE A.l Pros and cons of imaging techniques. Different imaging modalities have different resolution. 
While some approaches have a very high temporal resolution but a low spatial resolution, other modalities 
have an opposite relation. Source: Thomas Ramsoy, Daniela Balsler, and Olaf Paulson, with permission. 


systematic study of the relation between brain and 
behavior in humans. The new neuroimaging tech¬ 
niques introduced toward the end of the 20th century 
measure changes in brain activity that correlate with a 
change in behavior. Current neuroimaging equipment 
can measure the changes in activity of brain volumes 
smaller than 1 mm 3 and as fast as tens of milliseconds 
(Figure A.l). The popularity of these new imaging 
methods is illustrated by the continuous increase in 
the percentage of studies that use them. For example, 
in 2005 every fifth published brain study was indexed 
by PubMed under the keywords PET (positron emis¬ 
sion tomography) or MRI (magnetic resonance imag¬ 
ing) (Figure A.2). 

The clinical-pathological correlation and the newer 
brain imaging methods both tell us about the functional 
organization of the brain, but fundamental differences 
separate these methods. Thus, with the clinical-patho¬ 
logical correlation, we learn what deficits occur if a 
given brain region is injured, i.e. for which functions a 
given brain region is essential. On the contrary, with 
PET and fMRI (functional MRI) we observe which 
regions are activated in a given task as compared with a 
control condition. Thus, from the clinical-pathological 
correlation, we know that Wernicke's and Broca's areas 


PET and MRI studies 



Year 


FIGURE A.2 The dramatic increase of neuroimaging in brain 
science. In 2005, every fifth published brain study was indexed by 
PubMed under the keywords PET or MRI. Source: Thomas Ramsoy, 
Daniela Balsler, and Olaf Paulson, with permission. 

are essential for normal language function, but in any 
speech task, however, both sensory and motor areas 
involved in articulation will also be activated. For these 
reasons, sophisticated design of paradigms becomes 
essential in modern brain mapping. The task and con¬ 
trol condition should, in principle, be identical except 
for the single psychological function whose location 
we want to map. It is practically very difficult, if not 
impossible, to control perfectly all behavioral param¬ 
eters. For instance, in a paradigm designed to identify 
the neural correlate of spatial attention by presenting 
visual stimuli at either a cued or an un-cued location. 
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a change in activity in a brain area may as well reflect 
involuntary eye movements or the subject's effort to 
prevent a saccade towards the stimulus. 

1.2 Recording brain activation 

The brain is activated when we are engaged in solv¬ 
ing a task, thinking, speaking, purposefully moving, 
etc. Such activation is accompanied by changes in the 
hemodynamics in the activated brain regions. This 
forms the basis for several functional brain mapping 
studies including the now dominating fMRI methods. 
In the following, a brief overview of some of the mile¬ 
stones in this field will be given. 

Changes in hemodynamics during sensory, motor, 
and cognitive processes were observed about 130 
years ago by Mosso in Italy (Mosso, 1881). He had a 
patient with a skull defect that allowed him to make 
external recordings of the 'brain pulse'. He saw 
changes in the brain pulse not only when the church 
bells were ringing but also when the man said a 
prayer even if he did it silently without words. About 
10 years later, Roy and Sherrington (1890) proposed 
that there was a relation between the brain's function 
and its perfusion (blood flow). This theory has indeed 
been shown to be valid and has formed the basis for 
extensive research. 

In the late 1920s Fulton (1928) observed a patient 
with an intracranial arteriovenous malformation, a condi¬ 
tion where lesions of the cerebral vasculature develop 
such that blood flows directly from the arterial sys¬ 
tem to the venous system without passing through a 
capillary system. The patient complained of an abnor¬ 
mal sound (bruit) in the head, which was louder dur¬ 
ing reading. The bruit and its aggravation could be 
recorded, demonstrating that some kind of cerebral 
hemodynamic changes had to occur during reading 
(at least in this patient). In the late 1950s, it was dem¬ 
onstrated that stimulation of the brainstem resulted 
in activation in the EEG (electroencephalograph) as 
well as in cerebral blood flow increase (Ingvar and 
Soderberg, 1956,1958). 

In the mid-1960s. Cooper (Cooper et ah, 1966) 
observed an increase of the oxygen tension locally in 
the brain during activation in patients undergoing 
surgery. This study further strengthened the fact that 
hemodynamic changes had to occur during cerebral 
activation. Moreover, this was the first study illustrat¬ 
ing a change in the coupling between flow and metab¬ 
olism during activation. 

A milestone in investigation of regional cere¬ 
bral function was achieved in the early 1960s when 


Lassen and Ingvar introduced the intra-arterial injec¬ 
tion method using radiolabeled inert gases. In their 
first studies, they measured the clearance of the beta¬ 
imaging isotope 85 crypton from the exposed surface of 
the brain in experimental animals (Lassen and Ingvar, 
1961). Soon the method was adapted for investigation 
in humans using the gamma-emitting isotope 133 xenon, 
which could be detected through the intact skull. 
Using multiple external scintillation detectors (sen¬ 
sors measuring ionizing radiation), it became possible 
to measure blood flow from as many as 250 regions. 
The method was used in the 1970s abundantly for 
functional localization in the brain and was, indeed, 
the first method used for functional brain mapping 
(Lassen et al., 1978). One drawback of this method 
was that the tracer had to be injected directly into the 
internal carotid artery, limiting studies to patients who 
were undergoing angiograms for diagnostic purposes. 

In the 1970s, Sokoloff (Sokoloff et al., 1977) intro¬ 
duced the 14 C-deoxyglucose method for measurement 
of the metabolism of glucose in regions of experi¬ 
mental animals' brains. Soon the method was further 
developed for PET studies in humans with fluorinated 
deoxyglucose (FDG) labeled with the positron emitter 
18 F. It is known that changes in blood flow and in glu¬ 
cose metabolism are coupled. Although measurement 
of a regional cerebral glucose metabolism in many 
aspects represents a milestone, it has not turned out to 
be very useful for measurement of the functional cere¬ 
bral activation due to the limited time resolution (15-45 
minutes) as compared to flow measurement with PET 
and fMRI to be discussed later in this chapter. 

The establishment of PET in the 1980s and of 
fMRI in the 1990s provided new non-invasive meth¬ 
ods suitable for studies of regional changes in the 
brain's hemodynamics as an expression of changes in 
regional cerebral activity. The non-traumatic nature 
of these methods has allowed investigation of not 
only patients but also normal subjects and thereby 
expanded research in functional cerebral activation. 

2*0 METHODS 

PET and fMRI are today the dominant methods for 
investigation of functional cerebral activation. Single 
photon emission computer tomography (SPECT) 
has features similar to PET, but has a lower resolu¬ 
tion and is therefore less suitable for studies of func¬ 
tional activation. Other methods use the electrical and 
magnetic signal from the working brain produced 
by the activity in the neurons. These methods are 
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(b) 



FIGURE A.3 The BOLD signal to finger stimulation. When 
the right finger is stimulated with a mild electrical shock, the 
change in blood flow can be picked up on the left hemisphere in the 
somatosensory body map. This is the blood oxygen-level dependent 
or BOLD response of fMRI. (a): Averaging brain slices, (b): The blood 
oxygenated-level dependent signal change (hemodynamic response) 
after the neural activation. Source : McConigle in Frackowiak et al., 
2004. 


electroencephalography (EEG) and magnetoencepha¬ 
lography (MEG) and feature an excellent time reso¬ 
lution of about 10 milliseconds at the cost of limited 
localizing ability Research using EEG has gained new 


interest when combined with fMRI, because it takes 
advantage of both the spatial resolution of fMRI and 
the temporal resolution of EEG. MEG is a research 
field which is currently expanding. 
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All these methods manipulate behavior and measure 
a change in brain activity. A complementary category 
of methods, including lesion studies or transcranial 
magnetic stimulation (TMS), allows manipulation of 
the brain in order to measure the effect on behavior. 
TMS induces an electric current in the brain whose 
effect is either stimulatory or inhibitory for the neuro¬ 
nal activity, depending on the parameters of the cur¬ 
rent induced. Using such a Virtual lesion' mode, this 
method can reduce the excitability of a brain area with 
effects that mimic a brain lesion. TMS is a promising 
tool for confirming brain-behavior links identified with 
functional imaging. If an activated brain area is indeed 
essential for task performance, then its inactivation 
should change the subject's behavior during the task. 

2*1 Electroencephalography (EEG) 

Electroencephalography is a neurophysiological mea¬ 
surement of electrical activity in the brain. This record¬ 
ing is performed by placing electrodes either on the 
scalp or directly on the cortex. The resulting brainwave 
output is referred to as an electroencephalogram (EEG), 
originally named by the German psychiatrist Hans 
Berger (Berger and Gloor, 1969). Work on EEG tech¬ 
niques had already been performed by Richard Caton 
(Caton, 1875) and Vladimir Vladimirovich Pravdich- 
Neminsky (Pravdich-Neminsky, 1913). EEG is today 
used in clinical settings to assess brain damage, epi¬ 
lepsy, and other brain disorders. In many jurisdictions 
around the world it is used to assess brain death. It is 
also a cognitive neuroscience research tool due to its 
superior temporal resolution and its results are often 
compared to other brain recording techniques. 

The EEG ranges from several to about 75 pV in the 
awake and healthy individual. The EEG signal as such 
is mostly attributable to graded postsynaptic poten¬ 
tials occurring in the cell body and large dendrites. 
The pyramidal cells of layers 3 to 5 are the major con¬ 
tributing units to the signal and these neurons are 
synchronized by rhythmic discharges from thalamic 
nuclei. The degree of synchronization of the underly¬ 
ing cortical activity is reflected in the amplitude of the 
EEG. Not surprisingly, most of the signal recorded in 
the EEG stems from the outer cortical layers near the 
electrode and the folded organization of the cortex 
in humans contributes to the electrical summation of 
neuronal signals rather than mutual cancellation. 

The EEG recorded at the scalp represents a passive 
conduction of currents produced by summating activity 
over large neuronal aggregates. Regional desynchroni¬ 
zation of the EEG reflects increased mutual interaction 


of a subset of the population engaging in 'cooperative 
activity' and is associated with decreases in amplitude. 
Thus, from the raw EEG alone it is possible - even for 
the untrained eye - to determine the level of synchrony. 
To the trained eye, it is also possible to see pathologi¬ 
cal patterns following states such as epilepsy. As can 
be seen in Figure A.4, it is possible to make out some 
basic differences in healthy and pathological brain 
activation. 

In general, we can think of the EEG as two kinds 
of measures. First, spontaneous activity is the activity 
that goes on continuously in the living individual, as 
measured on the scalp or directly on the cortex. The 
measurement of this signal is what we call the enceph¬ 
alogram, which can be thought of as a measurement 
of electrical signals within a time window, but with 
no additional time factors. The amplitude of the EEG 
is about 100 pV when measured on the scalp and about 
1-2 mV when measured on the surface of the brain 
(intracranial recording). The bandwidth of this signal 
is from under 1 Hz to about 50 Hz. Figure A.4 displays 
different kinds of spontaneous EEG activity. Today, the 
EEG is used extensively for clinical purposes, especially 
in testing for epilepsy, but recently, the combination 
with imaging methods with high spatial resolution, 
such as fMRI, has led to renewed interest in the EEG. 

The second kind of EEG measure, evoked potentials 
(EP) and event-related potentials (ERP), are components 
of the EEG that arise in response to a stimulus (e.g. 
auditory, somatosensory, or visual input). Such sig¬ 
nals are usually below the noise level and not nor¬ 
mally possible to distinguish in the raw EEG output. 
In order to see the effects of the stimulus, one must 
apply a train of similar stimuli and then average the 
signal for all these epochs. In this way, the signals 
recorded at the time of a stimulus are grouped into 
one category. Averaging the signal for a time frame 
cancels out any spontaneous fluctuation in the EEG 
and it is therefore possible to make out and display 
the signal intensity within a time frame common to all 
stimulus epochs. So, for a visual stimulus, such as a 
word, we get different waves of increase and decrease 
in signal intensity (Figure A.5). Each of these com¬ 
ponents is today identified as a relevant indicator of 
cognitive processing states. For example, EEG record¬ 
ings of a subject reading a word show early increases 
or decreases indicative of early visual processing and 
the drawing of attention to the stimulus and later we 
find a significant peak after approximately 400 milli¬ 
seconds thought to indicate semantic processing. This 
component is called the P400, where P = a positive 
signal change. 
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FIGURE A.4 The raw EEG in different states, (a) The record of a healthy and awake individual shows different low-amplitude and 
high-frequency bands of activation, indicative of rapid communication between ensembles of neurons, (b) When going to sleep the brain can 
operate at several different levels of sleep, ranging from rapid eye movement (REM) sleep to stages of deep sleep. Deep sleep stages display 
high-amplitude and low-frequency patterns, which indicate lower cooperation between and within brain areas, (c) Finally, epileptic seizures 
show up clearly on the EEG as both high-amplitude and relatively high-frequency patterns on all recorded channels. This demonstrates that 
most parts of brain are affected by the seizure. This pattern of activity is indicative of areas that get into a mutual loop of activation, eventually 
leading to chaos and non-information processing in the brain. Source : Thomas Ramsoy, Daniela Balsler, and Olaf Paulson, with permission. 


An example of a well-known component using ERP 
is called mismatch negativity (MMN), sometimes also 
called the mismatch field. Consider that you are listen¬ 
ing for a stream of similar beep sounds, and suddenly 
the sound changes. The brain responses to this devi¬ 
ant sound, based on differences in pitch, duration, or 
loudness, can be elicited regardless of whether the 
subject is paying attention to the sequence. 

At the ERP curve, this event occurs with sources in 
the primary and nonprimary auditory cortex and a 


typical latency of 150-250 ms after the onset of the devi¬ 
ant stimulus. Although the exact nature of the MMN 
is not well understood, it is thought to reflect an auto¬ 
matic and early response to violation of an automati¬ 
cally formed, short-term neural model or memory 
trace of physical or abstract environmental regularities 
(Naatanen and Winkler, 1999; Naatanen, Paavilainen, 
Rinne, and Alho, 2007). Alternatively, it is possible that 
the MMN reflects changes in neural ensembles, and that 
the response of newly recruited networks, compared 
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(b) (d) 

FIGURE A.5 The raw electroencephalogram and two ways to analyze it. (a) A number of electrodes are placed on specific places around 
the scalp according to a prespecified system, (b) The readouts from each of these electrodes, or channels, can be plotted as separate curves over 
time and make up the raw encephalogram. Events such as sound stimuli can be plotted on the same time curve, both to give an idea of trends 
in the raw EEG, but also to be used in the later averaging of the data. The events are shown for illustrative purposes, (c) The event-related 
analysis of the raw EEG is made by averaging the EEG signal changes at the time interval around each stimulus event. By averaging the signal 
spontaneous activation is cancelled out, while the signal change common to all stimulus events will be visible. In this way, several components 
have been determined. These include early positive or negative peaks thought to involve subcortical activations (e.g. from the brainstem) and 
later onsets thought to involve more elaborate, cognitive processes. An example is the P400 component (P400 = positive peak around 400 mil¬ 
liseconds after stimulus onset, not shown here), which is thought to be an indicator of semantic processing. Finally, more recent developments 
have made it possible to analyze the data both in terms of the EEG bandwidths during a period and the relative spatial localization of these, 
(d) The EEG power spectrum topography shows the average bandwidths at rest. As can be seen, delta activity is focal at the prefrontal site and 
maximal at the AF8 electrode site. Theta activity is found at the frontal midline area, maximal at the FCz electrode site. Alpha-1 activity was 
centered on the parietal-occipital midline area and maximal at the P08 electrode site. Alpha-2 activity was focal at the occipital area and maxi¬ 
mal at the P04 electrode site. Beta activity was highly diffused across the scalp and a maximal site at the FCz electrode site was selected. 
Gamma activity was focal at bilateral temporal areas and maximal at the FFT7h electrode site. Source : (a) and (d) Andrew et al., 2006; (b) and (c) 
chapter authors, with permission. 
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BOX A. 1 The electromagnetism of meditation 


A very interesting study by Lutz et al. (2004), using EEG, 
focused on the brain activity of experienced Buddhist 
meditators. In this study, EEG signals were recorded in 
expert meditators and meditation-naive control subjects 
during normal resting phases and during different stages 
of meditation. Three task stages were used: baseline rest, 
meditative state, and a pause stage between meditative 
states. Lutz et al. found that, during meditation, the group 
of experienced meditators had a dramatically higher 
level of gamma-band oscillations. The researchers also 


found a long-distance phase synchrony between frontal 
and parietal areas in the brain. From these results. Lutz 
et al. speculate that meditative training enhances the inte¬ 
gration of distant brain areas. Interestingly, the results 
also showed that the brain activation even at rest before 
meditation practice differed between the expert and 
naive groups. This indicates that substantial meditative 
experience can alter the workings of the brain, although 
at present we can only speculate at the precise cause and 
effect relationships. 



(b) (d) 

FIGURE A.6 Measuring the brain effects of meditation: the study of Lutz et al. (2004) shows that the absolute gamma power 
during mental training is much higher in practicing meditators (right) as compared to non-meditating controls (left). The color scale 
indicates the percentage of subjects in each group that had an increase of gamma activity during the mental training. Source : Lutz 
et al ., 2004. 


to networks that have processed the repeated stimulus, 
respond more vigorously (Naatanen, 1992; Jaaskelainen 
et al., 2004). 

The ERP shows how it is possible to identify spe¬ 
cific activation patterns within the brain according to 
the changes in signal intensities across time. Although 
EEG is traditionally thought of as superior in temporal 
resolution but with poor spatial resolution, continu¬ 
ous technical developments have improved the spatial 
resolution dramatically. Today, EEG is performed by 
applying hundreds of electrodes on the scalp, typi¬ 
cally by using a head cap with predefined positions. 


As a research tool, EP/ERP can thus provide valuable 
information about both the precise timing and now 
also the gross cortical distribution of the activity gen¬ 
erated during mental activity. 

2.1.1 Intracranial EEG and deep electrodes 

Not all EEG recordings are done from the outside of 
the skull. In some cases, it is also possible to record the 
electrical properties directly from the brain itself. There 
are, in general, two ways of doing this: by laying an 
intracranial grid of electrodes on the surface of a part of 
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the brain; or by using deep electrodes directly into the 
brain. Both methods are invasive and are only applied 
in humans as part of a clinical evaluation, typically in 
the search for epileptic foci for surgical planning. In 
this clinical procedure, the surgeons need to know 
not only where the epileptic seizure initiates, but also 
where other important cognitive functions are located. 
If possible, surgeons avoid ablating brain tissue that 
is involved in a cognitive function such as memory or 
language. Therefore, the patient needs to be awake dur¬ 
ing the test, both in order to respond to the set of cogni¬ 
tive tasks that are applied and, in order to report any 
changes occurring during the test. While the electrodes 
are implanted, researchers are sometimes allowed a 
limited time to do scientific testing on these patients. 

In a study of intracranial deep electrodes in 
humans, Quiroga and colleagues (Quiroga et al., 
2005) recorded the response of single neurons in the 
medial temporal lobe when subjects saw images of 
faces and objects. Remarkably, a subset of the neurons 
responded selectively to strikingly different pictures of 
specific famous people, landmarks, or objects. In other 
words, they found that one neuron responded specifi¬ 
cally to pictures of actress Jennifer Aniston, another 
neuron responded significantly more to pictures of 
actress Halle Berry, and yet another neuron responded 
most to the Sydney Opera House (Figure A.7). This 
finding demonstrates an invariant, sparse, and explicit 
neural code that may be important in turning visual 
percepts into memories. 

While deep electrodes are implanted in the brain, 
it is possible to reverse the electrical current and 
stimulate the same area of the brain. In this way, one 
may grossly activate a highly specific region of the 
brain. By using this method, Barbeau and colleagues 
(Barbeau et al., 2005) found a difference in the subjec¬ 
tive reports of subjects when stimulating the hippo¬ 
campus or the adjacent perirhinal cortex, both areas 
within the medial temporal lobe. When stimulating 
the hippocampus, their patient reported full-blown 
episodic memories in rich detail and with scenic and 
episodic content. When stimulating the perirhinal cor¬ 
tex (Figure A.8), the patient reported recollection of a 
specific object that later developed into a full-blown 
episodic memory. While this study is limited by only 
using a few experimental stimulations and therefore 
should be interpreted with caution, the results indi¬ 
cate that areas within the medial temporal lobe play 
different roles in memory and recall. It seems that 
the hippocampus is involved in the recall of full epi¬ 
sodic memories, while the perirhinal cortex is more 
involved in specific object memory. 


Intracranial imaging is also done on non-human 
primates such as the macaque monkey. This ani¬ 
mal 'model' of the human cognitive system allows 
scientists to further their exploration of the neural 
correlates of cognitive functions. Important discover¬ 
ies have been made using this approach, such as the 
findings of mirror neurons made by Gallese and col¬ 
leagues (Gallese et al., 1996). 

Together, EEG presents a whole range of different 
recording (and stimulation) approaches that each makes 
significant contribution to the study of brain-mind rela¬ 
tionships. With its superior temporal resolution, the 
EEG provides valuable information to combine with 
other imaging techniques that have a high spatial reso¬ 
lution. We will return to the combination of such data at 
the end of this appendix. 


2*2 Magnetoencephalography (MEG) 

Magnetoencephalography (MEG) is the measurement 
of the magnetic fields produced by electrical activity 
in the brain, usually recorded from outside the skull. 
It is a highly interesting tool for investigation of func¬ 
tional activation of and connectivity in the brain. The 
spatial resolution with the most advanced instruments 
comes down to a few millimeters and the temporal 
resolution is, as with EEG, down to milliseconds. This 
allows us to record how activation spreads from one 
region to another. 


2.2.1 The principles of MEG 

The physical principles of MEG are based on the old 
observation from H. C. 0rsted from 1820 that an elec¬ 
trical current in a wire will generate a surrounding 
circular magnetic field. Since impulses propagating 
in the brain are generated by electrical currents, an 
abundant amount of small local magnetic fields will 
be generated. The magnetic field from a single neuron 
is far below detection level; however, the combined 
fields from a region of about 50 000 active neurons can 
give rise to a net magnetic field that is measurable. 

Let us for simplicity consider a single electrical wire 
which will be surrounded by a circular magnetic field 
orthogonal to the electrical current. If it corresponds to 
an electrical current in the brain, an electrical dipole, 
parallel to the surface of the skull, then the magnetic 
field will exit the skull on one side of the current and 
re-enter the skull on the other side. Changes in the elec¬ 
trical current in the brain and in the magnetic field will 
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FIGURE A. 7 Quiroga and colleagues (Quiroga et al, 2005) found a neuron in the left posterior hippocampus that selectively 
responded to different views of the actress Jennifer Aniston. Responses in 30 of a total of 87 images are shown (inset: the responses 
to images of Jennifer Aniston). Numbers indicate the image number; graph indicates number of neural spikes recorded. Similar 
findings were made in other neurons for other stimuli, including the actress Halle Berry and the Sydney Opera House. Source: 
Quiroga et al, 2005. 


induce an electrical current in a circular lead placed 
parallel to the surface of the skull. The induced cur¬ 
rent will change direction if the lead is moved from 
one side to the other of the current in the brain 
(Figure A.9). If, by contrast, an electrical current in the 


brain is perpendicular to the surface of the skull, then 
no magnetic gradients are produced outside the skull 
and an external lead will remain silent. 

The electromagnetic signals in the brain derive 
from the net effect of ionic currents in the dendrites 
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FIGURE A. 8 The stimulation of perirhinal cortex in object 
memory, (a) Image reconstruction of the patient's brain with the site 
of implantation of each electrode. The green line shows the implanta¬ 
tion site of the electrode running through the perirhinal region. Red 
and blue dots indicate other electrode implantation points in the same 
brain, (b) A CT scan showing contact location of the electrodes (white 
dotted lines), (c) Axial MRI structural scan showing the thin trace left 
by the electrode (white line), (d) Coronal MRI scan showing the stim¬ 
ulation site (green dots) in the depth of the occipitotemporal sulcus 
of the right temporal lobe, an area that corresponds to the perirhinal 
region (see text for discussion). Source : Barbeau et at., 2005. 



FIGURE A.9 Magnetoencephalography and its analyses. The subject is placed in the scanner that has a large set of shielded 
sensors. The signals themselves derive from the net effect of ionic currents flowing in the dendrites of neurons during synaptic 
transmission and in the extracellular medium as return currents (bottom). Action potentials do not produce an observable field 
because the currents associated with action potentials flow in opposite directions and the magnetic fields cancel out. Inset left: 
Magnetic fields following painful (epidermal) stimulation where (a) shows the recorded data; (b) and (c) display residual magnetic 
fields obtained after filtering the somatosensory processing signals from the recorded data. The bottom two lines show the time 
course of the source strengths during the painful stimulation. Inset right : Source locations of the MEG data overlaid on MR images. 
Source : 4D Neuroimaging, San Diego, with permission. 
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of neurons and in the extracellular space as return 
currents during impulse propagation and synaptic trans¬ 
mission. Action potentials do not produce significant 
fields as the currents associated with action potentials 
flow in opposite directions, canceling out the magnetic 
fields. 

From the above considerations on the direction of 
the electrical dipoles and generated magnetic fields, 
it appears that it is the neurons located in the wall of 
the sulci of the cortex with orientations parallel to the 
surface of the head that project measurable portions 
of their magnetic fields outside of the head. Thus, 
neurons on the top and in the bottom of the sulci will 
have an orientation which yields magnetic fields with 
minimal gradients outside the skull and will not be 
recordable. Still, deviation of the radial direction of the 
convex source by only 10 to 20 degrees can be enough 
to give a detectable signal. Therefore, it seems likely 
that especially the convex sources near the skull and 
thus near the recording apparatus contribute signifi¬ 
cantly to the MEG signals. 

2.2.2 M EG recording 

The magnetic signal emitted from the working brain is 
extremely small, a few femto-Teslas (17 fT = 10~ 15 T 1 ). 
Therefore, extremely sensitive and expensive devices 
such as the superconducting quantum interference 
device (SQUID) are used. The SQUID is an ultrasen¬ 
sitive detector of magnetic flux. It acts as a current- 
to-voltage converter that provides the system with 
sufficient sensitivity to detect neuromagnetic sig¬ 
nals. In order to record these weak magnetic fields, 
shielding from external magnetic signals, including 
the Earth's magnetic field, is necessary. An appropri¬ 
ate magnetically shielded room can be constructed 
from mu-metal, which is effective at reducing high- 
frequency noise, while noise cancellation algorithms 
reduce low-frequency common mode signals. With 
proper shielding, the SQUID acts as a low-noise, 
high-gain current-to-voltage converter that provides 
the system with sufficient sensitivity to detect neu¬ 
romagnetic signals of only a few femto-Teslas in 
magnitude. 

The first detection of magnetic rhythm from the 
brain dates back nearly 40 years and used an induc¬ 
tion coil magnetometer in a magnetically shielded 
room (Cohen, 1968). Modern systems have now up 


to about 300 SQUID channels placed around the head 
and have a noise level of around 5 to 7 femto-Teslas. 
This has to be compared to an overall magnetic field 
of the brain of around 100 to 1000 femto-Teslas. 

2.2.3 Data analysis 

The primary technical difficulty with MEG is that the 
problem of inferring changes in the brain from magnetic 
measurements outside the head (the 'inverse prob¬ 
lem') does not, in general, have a unique solution. The 
problem of finding the best solution is itself the sub¬ 
ject of intensive research today. Adequate solutions 
can be derived using models involving prior knowl¬ 
edge of brain activity and the characteristics of the 
head, as well as localization algorithms. It is believed 
by some researchers in the field that more complex 
but realistic source and head models increase the 
quality of a solution. However, this also increases the 
opportunity for local minima and potentially makes 
the numeric conditioning of the system worse, thus 
increasing the effects of model errors. Many experi¬ 
ments use simple models, reducing possible sources 
of error and decreasing the computation time to find 
a solution. Localization algorithms make use of the 
given source and head models to find a likely location 
for an underlying focal field generator. An alterna¬ 
tive methodology involves performing an indepen¬ 
dent component analysis first, in order to sort out the 
individual sources, and then localizing the separated 
sources individually. This method has been shown to 
improve the signal-to-noise ratio of the data by cor¬ 
rectly separating non-neuronal noise sources from 
neuronal sources, and has shown promise in segregat¬ 
ing focal neuronal sources. 

Generally, localization algorithms operate by suc¬ 
cessive refinement. The system is initialized with 
a first guess. Then a loop is entered, in which a for¬ 
ward model is used to generate the magnetic field that 
would result from the current guess and the guess is 
then adjusted to reduce the difference between this 
estimated field and the measured field. This process is 
repeated until a convergence between estimated and 
measured field is reached. 

Another approach is to ignore the inverse problem 
and use an estimation algorithm to localize sources. 
One such approach is the second-order technique 
known as synthetic aperture magnetometry (SAM), 


1 The tesla is the value of the total magnetic flux (a magnet's 'power') divided by area. Hence, reducing the affected area will generally increase 
the magnetic flux density. The tesla is a unit to define the intensity (density) of a magnetic field. The Earth's magnetic field at latitude 50° is 
5 |iT (5.8 X 10 -5 T) and on the equator at a latitude of 0° is 31 |iT (3.1 X 10 -5 T). 
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FIGURE A. 10 The magnetic gradient of the neurons at the top 
and bottom of a sulcus do not have an orientation that maximizes 
their contribution to the MEG recording. The magnetic changes 
occurring in neurons on the sulci walls are better measurable by the 
MEG apparatus. Source : Adams et al., 2004. 


which uses a linear weighting of the sensor chan¬ 
nels to focus the array on a given target location. 
This approach, also known as 'beamforming', has an 
advantage over more traditional source localization 
techniques because most sources in the brain are dis¬ 
tributed and cannot be well described with a point 
source such as a current dipole. 

A solution can then be combined with MRI images 
to create magnetic source images (MSI). The two sets 
of data are combined by measuring the location of a 
common set of fiducial points marked during MRI 
with lipid markers and marked during MEG with 
electrified coils of wire that give off magnetic fields. 
The locations of the fiducial points in each data set 
are then used to define a common coordinate sys¬ 
tem so that superimposing ('co-registering') the 
functional MEG data onto the structural MRI data is 
possible. 

A criticism of the use of this technique in clinical 
practice is that it produces colored areas with defi¬ 
nite boundaries superimposed upon an MRI scan: the 
untrained viewer may not realize that the colors do 
not represent a physiological certainty, because of the 
relatively low spatial resolution of MEG, but rather a 
probability cloud derived from statistical processes. 
However, when the magnetic source image corrobo¬ 
rates other data, it can be of clinical utility. 


2.2.4 Relation to other recording modalities 

MEG has been in development since the 1970s but has 
been greatly aided by recent advances in computing 
algorithms and hardware and promises good spatial 
resolution and extremely high temporal resolution 
(better than 1 ms). Since MEG takes its measurements 
directly from the activity of the neurons themselves, 
its temporal resolution is comparable with that of 
intracranial electrodes. MEG's strengths complement 
those of other brain activity measurement techniques 
such as EEG, PET, and fMRI whose strengths, in turn, 
complement MEG. Other important strengths to note 
about MEG are that the biosignals it measures are 
not distorted by the skull, as in EEG (unless magnetic 
metal implants are present) and that it is completely 
non-invasive, as opposed to PET. 

In research, the primary use of MEG is the mea¬ 
surement of time courses of activity as these courses 
cannot be measured using fMRI. Due to various tech¬ 
nical and methodological difficulties in localization of 
sources using MEG, its use in creating functional maps 
of human cortex plays a secondary role, as verifica¬ 
tion of any proposed maps would require verification 
using other techniques before they would be widely 
accepted in the brain mapping community. 

The clinical uses of MEG have until now essen¬ 
tially been limited to investigation of special cases for 
detecting and localizing seizure activity in patients 
with epilepsy and in localizing cortical pathology for 
surgical planning in patients with brain tumors or 
intractable epilepsy. 

The physiological basis of the neuroimaging signal 
is the coupling between regional cerebral activation 
and blood flow and, further, the uncoupling between 
flow and oxygen consumption during activation. Put 
simply, an increase in activity in the cerebral tissue 
leads to a blood flow increase that can be measured 
with PET. The increase in flow markedly exceeds the 
increase in oxygen consumption so that the concentra¬ 
tion of deoxyhemoglobin decreases. This local decrease 
in deoxyhemoglobin gives rise to the fMRI signal. 

The epileptic seizure can be considered as an 
extreme non-physiological activation where all neu¬ 
rons fire at maximal rate. In this condition, blood 
flow is markedly increased and oxygen consump¬ 
tion is also increased but much less so, resulting in an 
increase in oxygen content in the cerebral venous blood 
(Brodersen et al., 1973). This corresponds to the obser¬ 
vation by Cooper and co-workers, described earlier, of 
an increased oxygen tension in cerebral tissue. Raichle 
and Fox, in an elegant PET study, demonstrated that 
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BOX A.2 Phantom limb sensation and brain plasticity 


A good example of the application of MEG to study psy¬ 
chological or perceptual issues is a study done by Yang 
et al. (Yang et ah, 1994) (see also Ramachandran and 
Hirstein 1998; Ramachandran and Rogers-Ramachandran 
2000). These researchers studied the plastic changes that 
occur in the brain after amputation of a limb. Phantom 
limb sensations occur in almost all people who have 
a limb amputated. What this means is that people who 
lose their arm are still experiencing sensations from 
that arm, although it is not there! This phenomenon has 
been known throughout history - e.g. by Lord Nelson, 
whose phantom arm led him and others to think that the 
phantom was proof that the immaterial soul could exist 
without the physical body. Today, we are able to mea¬ 
sure changes occurring in the brain of those unfortunate 
people. 

It is possible to produce phantom limb sensations 
in these patients. For example, in an arm amputee, one 
may produce a phantom arm sensation by tickling the 
chin or jaw on the face on the same side as the amputa¬ 
tion. By doing this, Yang et al. found that those areas that 
would normally be activated by stimuli to an intact arm 
now became activated by stimuli to either the face or the 
amputation stump on the patient. These findings dem¬ 
onstrate that amputation leads to a remapping of the pri¬ 
mary somatosensory cortex and that one brain area can 
alter its functional connections, even in adults. Today, it 
is believed that amputation leads to three steps of change 
in the somatosensory cortex. First, amputation leads to 
a loss of input from the original sensory area. This then 
leads to functional changes in already existing connec¬ 
tions within the somatosensory cortex. Finally, as these 
functional changes continue, they will eventually lead to 
changes in the physical connectivity between neurons. 

Normally, adjacent areas that represent different body 
parts - e.g. face and hand areas - have tight interconnec¬ 
tions. This facilitates the representations to be specific, in 
saying both This is input from the hand' and This is not 
input from the face'. When an arm is amputated (or even 
disabled for a prolonged time), it leads to such a remap¬ 
ping of the somatosensory cortex. This was clearly dem¬ 
onstrated by Yang et al /s MEG study. 


during normal physiological activation of the brain 
blood flow, as measured by 15 0 2/ and glucose phospho¬ 
rylation, as measured by the 18 F-FDG, increase in par¬ 
allel, whereas oxygen consumption only increases to a 
minor extent (Fox and Raichle, 1986; Fox et ah, 1988). 

Theoretical calculations suggest that a large share 
of the metabolic energy is spent on action potential 
propagation along axon collaterals (Attwell and 
Laughlin, 2001). However, empirical studies using 
simultaneous electrophysiological and hemodynamic 
recordings by Lauritzen and co-workers (Lauritzen, 
2001) and by Logothetis and co-workers (Logothetis 
et al., 2001) have demonstrated that the increase in 


In the study by Yang and colleagues, stimulation of 
the face led to phantom sensations and was correlated 
to co-activation of both the face and hand representa¬ 
tion areas in the somatosensory cortex. A nice feature in 
this study is that the non-affected hemisphere serves as a 
control condition. That is, in addition to looking at other, 
non-amputated, subjects, we can also compare the activa¬ 
tion between the hemispheres. The study by Yang et al. 
is today considered to be among the very first studies to 
demonstrate plastic changes in the human brain. 



FIGURE A. 11 Normally, adjacent areas that represent 
different body parts, for example, the face and hand areas, are 
tightly interconnected. This helps make the representations 
highly specific, in saying both. This is input from the hand' and 
This is not input from the face'. When an arm is amputated (or 
even disabled for a long time), it leads to such a remapping of 
the somatosensory cortex. This was clearly demonstrated by 
Yang et al /s 1994 MEG study, where they reported that stimula¬ 
tion of the face led to phantom sensations in the missing (ampu¬ 
tated) hand, which correlated to co-activation of both the face 
and hand representation areas in the somatosensory cortex. 
Clearly, a remapping had occurred in this patient. Source: Yang 
et al, 1994. 


flow and BOLD (blood-oxygenation-level-dependent) 
signal reflects the increased synaptic activity and local 
field potentials in the dendrites, rather than a higher 
firing activity in the postsynaptic neurons. Thus, a 
release of stimulating as well as of inhibiting neuro¬ 
transmitters will result in an increased metabolic turn¬ 
over that increases blood flow and the BOLD signal. 

2*3 Positron emission tomography (PET) 

A modern PET scanner has more than 10000 scintilla¬ 
tion detectors registering the decay of positron emit¬ 
ting isotopes. When these isotopes decay, the emitted 
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positron will immediately meet a normal electron 
which merges and forms two gamma rays with energy 
of 511 keV 2 and an angle of precisely 180° relative to 
each other. When two detectors in the scanner regis¬ 
ter a photon of 511 keV exactly simultaneously, then a 
decay must have occurred on the line connecting these 
two detectors. At normal counting rates, the chance 
that two detectors simultaneously record two pho¬ 
tons originating from two different positron decays 
is unlikely. Thus, it becomes possible to reconstruct a 
three-dimensional picture of the distribution of the iso¬ 
tope in the field of view of the PET camera. 

The most used isotope compound for PET meas¬ 
urement of functional brain activation is H 2 ls O. ls O 
is a positron-emitting isotope with a half-life of only 
2 minutes. For that reason, the isotope has to be pro¬ 
duced in a cyclotron in the immediate vicinity of the 
PET camera. H 2 ls O is injected intravenously and will 
reach the brain regions in proportion to the blood sup¬ 
ply. Although not completely diffusible, water is highly 
diffusible across the blood-brain barrier and most of the 
water arriving at the vascular bed in the brain will dif¬ 
fuse into the tissue. Later washout of the radiolabeled 
water from the brain will take place, also in propor¬ 
tion to the perfusion. Thereby it becomes possible to 
calculate cerebral blood flow. A quantitative calcula¬ 
tion is possible; however, in many studies of functional 
activation only relative changes are recorded. Due to 
the fast decay of ls O, the radiation dose to the subject 
investigated is rather low, especially when using a PET 
scanner in 3D mode (three-dimensional recording with 
high counting efficiency). It is possible to make about 
24 activation studies with a radiation exposure of only 
approximately two times the yearly background radia¬ 
tion. Since PET relies on radioactive compounds, there 
are specific ethical limitations to how high a dose a 
subject can be given within a given period of time. As a 
result, weight is put on the radioactive property when 
choosing between compounds in a study. 

Another positron-emitting isotope which has been 
used for functional activation studies is 10 C in the form 
of 10 C carbon dioxide (Law et al., 2001). The half-life of 
10 C is only 19 seconds yielding a very low exposure for 
radiation and thus allowing for up to 64 PET scans with 
the same radiation exposure as for 12 scans using H 2 ls O. 
The short half-life of 10 C also allows repeated measure¬ 
ments with shorter intervals or following the activation 
during continuous infusion at the tracer. 10 CO 2 has only 
gained limited use, probably because an even higher 





FIGURE A.12 5-HT2A receptor binding as measured with 
PET and co-registered to a structural MRI image. Source : Adams 
et al ., 2004. 

time resolution without exposure to radioactivity can be 
obtained with fMRI. Still, the 10 CO 2 method might have 
special interest in areas where fMRI is hampered with 
susceptibility artifacts, as for instance close to the nasal 
sinuses, such as the orbitofrontal cortex. 

More recently, advances have been made toward 
functional imaging of neurotransmission; exciting 
prospects have recently emerged for in vivo monitoring 
of the brain's own signaling in terms of neurotransmit¬ 
ter release. A prime example is the PET demonstration 
of competition effects between endogenous dopamine 
and the radiolabeled dopamine D2 receptor antago¬ 
nist 11 C-raclopride or the SPECT tracer 123 I-IBZM (see 
Figure A. 13). Striatal radioligand binding decreased as 
the dopamine level increased under a successful video 
game session (Koepp et al., 1998). So far, the concept 
has primarily been used in conjunction with D2 recep¬ 
tor imaging and, in spite of attempts to identify PET 
tracers sensitive to, e.g. serotonin release (Pinborg et al., 
2004), a well-suited tracer still needs to be identified. 

23.1 Image analysis 

An image analysis approach frequently used to com¬ 
pare groups is the 'region of interest' (ROI) technique. 
In this, each subject's image is shown on a visual dis¬ 
play unit and a tracker ball or pen is used to outline 
regions of interest. The mean activity for each region 
for the experimental subjects would then be compared 


2 An electronvolt (eV) is the amount of kinetic energy gained by a single unbound electron when it passes through an electrostatic potential dif¬ 
ference of 1 volt, in vacuum. The kilo-electronvolt (keV) is 1000 eV. 






570 


APPENDIX: METHODS FOR OBSERVING THE LIVING BRAIN 


Baseline 


Stimulated 



FIGURE A.13 The dopaminergic synapse at baseline (left) and after pharmacological stimulation with amphet¬ 
amine. Bottom panel shows llC-raclopride binding levels (in a steady-state approach) as dopamine levels increase. 
Source: Laruelle, 2000. 


with that of the controls. The main drawback with 
this technique is that it takes no account of structural 
brain differences which might exist between subjects. 
Using a human operator to decide the boundaries of 
regions of interest on the functional image requires the 
assumption that the operator knows where to place the 
boundaries. On a functional image, the inference that 
structural boundaries can be identified is obviously a 
circular one. There are two solutions to this problem. 
The first is to obtain structural imaging data as well as 
functional imaging data for each subject. Then, data- 
processing techniques can be used to 'co-register' the 
imaging data on a voxel-by-voxel basis, such that a 
superimposed map for each subject is obtained. The 
PET image in Figure A. 12 shows a 3D rendered image 
of a co-registered PET and MRI image. 


2*4 Magnetic resonance imaging (MRI) 

MRI is an imaging technique used primarily in medical 
settings to produce high quality images of the inside 
of the human body. It is based on the principles of 
nuclear magnetic resonance, which is a spectroscopic 


technique used by scientists to obtain information 
about the chemical and physical properties of mol¬ 
ecules. As such, MRI started out as a tomographic 
imaging technique - a method for obtaining pictures of 
the interior of the body. Today, MRI has advanced far 
beyond this and now represents a battery of different 
approaches that can measure the structure, function, 
connectivity and chemistry of any part of the body. 

MRI is based on the absorption and emission of 
energy in the radiofrequency range of the electromag¬ 
netic spectrum. The human body is mostly made of 
fat and water - body tissues that have many hydrogen 
atoms. As such, the human body consists of about 65 
percent hydrogen atoms. These hydrogen nuclei form 
the very basis for the signal in MRI. A voxel is a vol¬ 
ume element that represents a value in 3D space. This 
is analogous to a pixel, which represents 2D image 
data. Voxels are frequently used in the visualization 
and analysis of medical and scientific data. In brain 
imaging, the brain is divided into a number of voxels 
and within each voxel one can do different kinds of sta¬ 
tistical testing. Each voxel (Figure A. 15) in the human 
body contains one or more tissues. Zooming in on the 
voxel reveals cells and within each cell there are water 
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BOX A.3 Altered neurochemistry in psychiatric disease 


In a study by Adams et al. (2005) patients suffering from 
obsessive-compulsive disorder (OCD) were scanned 
for the binding of serotonin in the brain (using the 
[ 18 H]altanserin binding to 5-HT 2 a receptors). OCD is a psy¬ 
chiatric condition affecting 2-3 percent of the population 
worldwide and it is classified as an anxiety disorder. The 
symptoms include recurrent, unwanted thoughts (obses¬ 
sions) and conscious, ritualized acts (compulsions), usually 
attempting to deal with the apprehension generated by 
the obsessions. OCD is often treated with a selective sero¬ 
tonin reuptake inhibitor (SSRI), a chemical compound that 
makes serotonin available in the synapse for a longer time. 

When comparing a group of OCD patients to an age- 
and gender-matched control group, Adams and her col¬ 
leagues found an increased [ 18 H]altanserin binding in the 
caudate nuclei. This indicates that there are more active 
5-HT 2 a receptors in the OCD brain than in healthy controls. 
Such an up-regulation of serotonin receptors is suggested 
to be a compensatory mechanism due to lack of serotonin 
in OCD patients. The serotonin network being affected 
includes structures such as the orbitofrontal prefrontal cor¬ 
tex, thalamus, caudate nucleus, and globus pallidus. 

The researchers then rescanned the patients after 
administering SSRI for at least 12 weeks, a treatment 
that would increase the amount of serotonin available 
in the OCD brain. The results now showed no difference 
between the control group and the patient group in the 
serotonin binding. In other words, there were not more 
active 5-HT 2A receptors in the OCD groups after treat¬ 
ment than in the control group. 


Caudate nucleus 5-HT2A receptors in OCD 



5-HT2A level 
(Adams et al., 2005) 

□ Control ■ OCD untreated 

FIGURE A. 14 5-HT 2 a receptor activation level in untreated 
OCD and healthy controls. 


PET studies like these provide important information 
about the neurochemistry of psychiatric disease and they 
have extended this understanding by providing new evi¬ 
dence about the neurochemical workings of psychiatric 
drugs such as SSRIs. 



FIGURE A. 15 Voxels as 
minimum boxes of brain space. 
A voxel of the brain. The voxel is 
a representation of a volume in 
three-dimensional space. In the 
brain, the resolution of the scanner 
determines how small the voxels 
can be. Parameters such as higher 
scanner field strengths increase 
the spatial resolution and hence 
the ability to represent separate 
structures in the brain. The brain 
voxel extracts the signal from one 
part of the brain, where the local 
molecular environment influences 
the magnetic response. The voxel 
chosen here is much larger than 
usual for MRI scans, for illustra¬ 
tive purposes. Source : Jones et ol., 
2002 . 
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molecules. Each water molecule consists of one oxygen 
atom and two hydrogen atoms. If we zoom into one of 
the hydrogen atoms past the electron cloud we see a 
nucleus comprised of a single proton. 

In a magnetic field, such as the MR scanner, the 
magnetic orientation of each hydrogen atom is aligned 
to the magnetic field and spins around this orientation 
(Figure A. 16(b) and (c)). If a brief electromagnetic (radio¬ 
frequency) pulse is applied, it temporarily distorts the 
atom's alignment to the magnetic field (Figure A. 16(d)). 
When the radiofrequency pulse ends, the atoms start 
to realign to the magnetic field, a process called relaxa¬ 
tion (Figure A.16(e)). It is during this phase that the 
atom loses its own energy by emitting its own energy, 
providing information about the environment. The 
relaxation occurs in two dimensions: Time-1 and Time- 
2. The realignment with the magnetic field is termed 
longitudinal relaxation and the time in milliseconds 
required for a certain percentage of the tissue nuclei 
to realign is termed Time V or Tl. This is the basis 
of Tl-weighted imaging, which produces the most 
well-known structural images in MRI. T2-weighted 


imaging relies upon local dephasing of spins following 
the application of a transverse energy pulse; this trans¬ 
verse relaxation time is termed Time 2' or T2. 

The Tl and T2 constants provide the basis for most 
medical imaging. In different parts of the body, such 
as the brain, different tissues alter the speed in which 
Tl and T2 relaxation occurs. The three most typical 
tissues of the brain are gray matter (GM), white mat¬ 
ter (WM), and cerebrospinal fluid (CSF). The influence 
of these tissues produces different signal intensities - 
contrast - that make it easy to distinguish between 
them. By varying different parameters during scan¬ 
ning, such as the rate and amplitude of the radiofre¬ 
quency pulse, or the time from excitation to recording, 
it is possible to highlight different properties of the tis¬ 
sues and their differences. 

This can clearly be seen in images that display the dif¬ 
ference between Tl- and T2-weighted images (Figure 
A. 17). As the images show, a brain tumor that can 
be hard to see on a Tl-weighted image is clearly vis¬ 
ible on the T2-weighted image. This is due to the fact 
that tumors contain more water and hence give rise to 



FIGURE A.16 The signal that makes up the MRI. (a) Outside the scanner the atoms are oriented at ran¬ 
dom in the brain, (b) When a subject is put into the scanner, the atoms align to the magnetic field of the scanner 
(B 0 ). However, the alignment is not perfect, since neighboring atoms influence each other, (c) At such baseline, 
the atom spins along the y-axis, i.e. the B 0 field, (d) When a radiofrequency (RF) pulse is applied the spin of the 
atoms is influenced and "pushed' down. This is a state of disequilibrium and during equilibration toward the 
B 0 field the atom releases energy that it received from the RF pulse, (e) The local milieu of the atom, i.e. whether 
it is in gray matter, white matter, bone, or cerebrospinal fluid, determines the speed of this relaxation. This is the 
basis of contrast in the MR image and thus what makes it possible to visualize the different tissues of the body. 
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a larger signal using T2-weighted imaging due to the 
slower longitudinal relaxation time. Since the longitu¬ 
dinal relaxation is fast in water, the extra water in the 
tumor produces no extra signal in Tl-weighted imaging. 

Structural scanning techniques are obvious choices 
when studying alterations in the brain due to aging, 
brain injury, or degenerative disorders such as 
Alzheimer's disease. Based on such imaging of brain 
morphology (structure), it is also possible to ask ques¬ 
tions about how the brain changes structurally as a 
result of development, aging and degenerative diseases. 
For example, Jernigan et al. (2001) demonstrated age- 
related changes in different parts of the cortex and 
cerebellum. By using the structural MRI images and 
drawing in the anatomical regions, the researchers 
were able to compare the relative size of these struc¬ 
tures across the age range. As Figure A. 18 illustrates, 
there are profound changes occurring in the brain as 
we age. Furthermore, there is a non-linear relation¬ 
ship between age and brain volume, something that 
contradicted the earlier views about the aging brain. 
Finally, a significant finding in this literature has been 
that there are regional differences in how the brain 
ages. 

2.4.1 Functional MRI 

While Tl- and T2-weighted images are superior at 
imaging the structure of the brain, MRI also offers 
ways to measure different functions of the brain. In 
general, there are two main approaches: BOLD fMRI 
and perfusion MRI. While the BOLD approach relies 
on a complex series of events that couple brain acti¬ 
vation to vascular changes and the relative level of 
regional oxygenated blood, perfusion MRI measures 



FIGURE A. 17 Tl- and T2-weighted images. Note that the 
brain tumor can be hard to see on a Tl-weighted image, while it is 
easy to see on the T2-weighted image - it is the round dark spot on 
the left of the brain image on the right. Source : Nakai et al., 2004. 


the cerebral blood flow (CBF) or cerebral blood vol¬ 
ume (CBV). 

BOLD fMRI is the most used and well-known way 
to assess brain activation with MRI. Brain activation 
changes the relative concentration of oxygenated and 
deoxygenated hemoglobin - blood with or without 
oxygen, respectively - in the local blood supply. While 
oxygenated blood is diamagnetic and does not change 
the MRI signal, deoxygenated blood is paramagnetic 
and leads to a drop in the MRI signal. If there is more 
deoxygenated blood in a region it therefore leads to a 
drop in the BOLD signal and more oxygenated blood 
in the region leads to a higher signal. The BOLD 
response can be thought of as the combination of four 
processes (Figure A.19): 

1 An initial decrease (dip) in signal caused by a com¬ 
bination of a negative metabolic and non-meta- 
bolic BOLD effect. In other words, when groups of 
neurons fire they consume more oxygen. When 
this happens, the local level of oxygenated blood 
drops and there is relatively more deoxygenated 
blood in that area. In addition, there is also a dila¬ 
tion of the blood vessels, which further increases 
the negative BOLD effect. 

2 A sustained signal increase or positive BOLD effect 
due to an increased blood flow and a corresponding 
shift in the deoxy/oxyhemoglobin ratio. When the 
neurons go back to a lower level of activation, their 
increased consumption of oxygenated blood drops. 
At the same time, influx of new oxygenated blood is 
increased due to the previous demand. As the blood 
oxygenation level increases, the signal continues to 
increase. This drop in demand of oxygenated blood, 
combined with a delayed supply of oxygenated 
blood, leads to a dramatic 'overshoot' of the relative 
amount of oxygenated blood. This abundance leads 
to the main effect of the BOLD signal. 

3 A sustained signal decrease which is induced by 
the return to normal flow and normal deoxy/oxy 
hemoglobin ratios. 

4 A post-stimulus undershoot caused by a slow recov¬ 
ery in cerebral blood volume. 

The 'initial dip' is thought to be the measure most 
closely related to the neural activation, since it relates 
to the first drop in signal intensity due to consumption 
of oxygenated blood. However, the signal changes 
at this stage are so small that they are mostly detect¬ 
able by the use of extra strong MRI scanners, which 
are both expensive and not yet generally approved for 
human testing. It is therefore the second phase - the 
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FIGURE A. 18 The brain ages differently in different brain regions. Top : Example of structural boundaries of the 
brain drawn as Regions of Interest. Note that the slices consist of both gray and white matter separately; white matter is 
presented with a lighter color. All white matter voxels that show elevated signal in a group of elderly relative to younger 
adults are shown in yellow. Bottom: Age-related changes in two regions of the brain in the age range 10-100 years, the 
thalamus and nucleus accumbens (indicated by arrow). Two major effects can be seen. First, both structures seem to have 
three time-related phases: (1) a precipitant loss of volume in both structures during adolescence and into young adulthood; 
(2) a flattening out and relatively stable state until late adulthood; (3) a late adulthood loss of volume. Second, the two 
structures show different trajectories in loss of volume. Sources: Top: Jernigan et al., 2001; bottom: Jernigan, with permission. 
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FIGURE A. 19 The blood oxygenated-level depen¬ 
dent signal simplified in four steps. Step 1: increased 
neural activation leads to an increase in the consumption 
of oxygen from the blood, leading to a lower level of oxy¬ 
genated blood and more deoxygenated blood, leading to 
a drop in the BOLD signal. Step 2: the vascular response 
to the increase in oxygen consumption leads to a dramatic 
increase in new, oxygenated blood at the same time as the 
oxygen consumption drops due to decreased levels of neu¬ 
ronal activation. Step 3: a normalizing of flow and deoxy/ 
oxyhemoglobin levels (not shown). Step 4: a post-stimulus 
undershoot caused by the slow recovery of blood volume. 


sustained signal increase - that is used in most BOLD 
fMRI studies. This signal is an indirect measure of 
neural activation, as it is the result of a delayed vas¬ 
cular overshoot of oxygenated blood, as a response of 
demand for oxygenated blood in a region. Although 
the BOLD response is an indirect and delayed mea¬ 
sure of neural activation, it has been shown to have a 
time resolution at the millisecond scale. In addition, 
the method has a very high spatial resolution at below 
1mm resolution. With recent technical advances, the 
resolution has become even smaller than this, bring¬ 
ing MRI to the sub-millimeter scale. 

There are different ways in which a study can 
be conducted using fMRI. In general, there are two 
main design categories - block designs and event- 
related designs. Following PET studies, the block 
design was the most used design in the earliest days 
of fMRI. Typically, a block design consists of a pre¬ 
sentation of stimuli as blocks containing many stim¬ 
uli of the same type. For example, a block design 
may be used for a sustained attention task, where the 
subject is instructed to press the button every time he 
or she sees an X on the screen among other letters. 
Typically, blocks are separated by equally long peri¬ 
ods of rest, although one may design block experi¬ 
ments without rest. A block design could be used if 
we were interested in seeing the difference between 
encoding of the identity versus the position of objects 
(Figure A.20). 

In an event-related design , stimuli are presented in a 
random or pseudo-randomized order. Here, individ¬ 
ual stimuli of various condition types are presented 


in randomized order, with variable stimulus onsets. 
This approach provides a means to look at differ¬ 
ent trial-relevant changes such as correct versus error 
responses. An event-related paradigm therefore lies 
closer to a traditional psychological experiment and 
it allows post-hoc analyses with trial sorting (accuracy, 
performance, response time, etc.). This design is more 
efficient, because the built-in randomization ensures 
that preparatory or anticipatory effects (which are 
common in blocks designs) do not confound event- 
related responses. An example of an event-related par¬ 
adigm can be seen in Figure A.21. 

When do you choose to use either a block or event- 
related design? In general, one can say that if you want 
to study state-related processes (as in 'encoding state' 
versus 'retrieval state'), then a block design would 
be the best choice. However, if you want to study 
item-related processes (as in correct versus incorrect 
responses), an event-related design is optimal. In certain 
experimental setups, it is possible to combine block and 
event-related designs, making it possible both to track 
overall differences in cognitive states, as well as being 
sensitive to differences from instance to instance. 

When considering the experiment design, another 
important issue concerns the states that are compared. 
There are several different contrasts one can choose to 
make. In some cases, especially in older neuroimaging 
studies, a comparison is made between a specific task 
and a rest period (see Figure A.20). Today, it is more 
usual to compare two or more active states. In general, 
we can distinguish between within-subject compari¬ 
sons and between-subject comparisons. In within-subject 
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FIGURE A.20 The block design. Intervals of separate trials (X and Y) are separated by periods of rest. Note 
that the order of X and Y trials is random. In such a design, X trials could be "reading letters' while Y trials could be 
'reading numbers'. 






Trial type 1 
Trial type 2 


FIGURE A.21 The event-related design. Unlike the block design the 
event-related design focuses on individual trials. Hence, the order of stimuli is 
much more random and is good for use in psychological designs where we do 
not want the subject to be able to predict the order of stimuli. An example para¬ 
digm could be to present stimuli at very brief durations and ask for each stimu¬ 
lus presentation whether the subject has seen the stimulus. Responses will vary 
according to the duration of the stimulus, as well as endogenous processes (e.g. 
inattention). The subjects are not able to predict their next response (or stimulus 
experience). 


BOX A.4 The BOLD nature of conscious perception 


What is the neural fate of stimuli that are consciously per¬ 
ceived? How do these processes compare to stimuli that 
are processed but not experienced? Are there situations 
where we get vague impressions that something was pre¬ 
sented yet we are unable to describe it? If so, how does 
the brain work under such perception? 

The neural correlates of consciousness have been 
studied for well over a decade using neuroimaging 
tools. Such findings converge on the idea that con¬ 
scious perception - as opposed to unconscious per¬ 
ception - involves a widespread network of areas in 
the brain (Baars, 2002b). This network is comprised 
of prefrontal and parietal cortices and very often tem¬ 
poral areas. It is also speculated that the spreading of 
information within this network occurs through cor- 
tico-thalamo-cortical loops, i.e. signals initiating in the 
cortex spread to widespread connections through the 
thalamic nuclei. 

Recently, behavioral studies have documented that 
the usual distinction between conscious and uncon¬ 
scious perception has to be reconsidered (Ramsoy and 
Overgaard, 2004). If subjects are allowed to make use of 
more elaborate responses about their experiences, they 
consistently acknowledge that some of their experiences 
can best be described as vague. This condition can be 
thought of as the detection of something being presented 


without the ability to identify what it was. One question 
thus becomes: how does this experience relate to con¬ 
scious and unconscious perception? Just as important, 
one may ask if there is a different neural substrate for 
such experiences. 

In a study by Christensen et al. (2006), this question was 
examined by presenting backward masked visual stimuli 
at different durations while scanning subjects with fMRI. 
This paradigm leads to different levels of experience of 
the stimuli and, from the experience of previous behav¬ 
ioral studies (Ramsoy and Overgaard, 2004), subjects 
were allowed to use a non-dichotomous rating scale to 
report experiences as not seen, vague, and clear experi¬ 
ences. Simple visual objects were presented for brief 
durations up to about 100 milliseconds. For each stimulus 
the subjects rated how clearly they had seen the object. 
This made it possible to compare unique activations for 
neural processes underlying clear and vague experiences. 

First, the comparison between reports of clear and no 
experience, i.e. the traditional difference between conscious 
and unconscious perception, led to the previously seen 
widespread network of brain areas. This included the 
parietal and prefrontal cortices, the temporal lobe (fusi¬ 
form gyrus), well known for its role in object perception 
(Figure A.22(a)). Conscious perception was also shown to 
involve large amounts of the thalami in both hemispheres. 


(Continued) 
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As such this finding supports a 'globalist' account of con¬ 
sciousness in the brain, stating that conscious perception 
requires a widespread network of brain areas. 

What, then, about vague experiences? When compar¬ 
ing reports of vague and no experience the researchers 
found activation in much of the same network only not 
as widespread, as they had found in conscious perception 
(Figure A.22(b)). By comparing states of clear experience 
to vague experiences, it was possible to identify the areas 
that are involved in turning detection into identification 


(Figure A.22(b)). From this, it seems that when we detect 
something, or have a vague feeling of seeing something, 
we use a widespread network of brain areas, with spe¬ 
cial activity found in the prefrontal cortices. If this vague 
perception changes into a fully conscious perception of a 
stimulus, the activity in the network increases dramati¬ 
cally both in spread and amplitude. As such, this study 
by Christensen et al. has provided novel and important 
new data to our understanding of conscious and uncon¬ 
scious perception. 





T 
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FIGURE A.22 An fMRI study by Ramsey and colleagues (Christensen et al., 2006) shows differing 
networks of activation for visual stimuli that are clearly perceived versus vaguely perceived. Top two rows 
(a) show the significant activity for clear perception (C) compared to no perceptual experience of the stimulus 
(N). Next two rows (b) show the activity for vague or glimpse-like (V) perceptual experience compared to N. 
Source: Christensen et al., 2006. 
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FIGURE A.23 Combining blocked and event-related designs is possible under 
certain circumstances. Here, analysis can focus either on the block level, e.g. com¬ 
paring encoding and recall activation, or it can use the behavioral data from a recall 
phase to look at the neuronal activation within the encoding blocks for the instances 
of successful encoding only (i.e. ignoring data for stimuli that were later forgotten). 
Source : Thomas Ramsoy, Daniela Balsler, and Olaf Paulson, with permission. 


TABLE A. 1 Different designs for neuroimaging studies 


Within-subject designs 


Examples 


Different processing states Reading words aloud versus self-generation of words 

Compare effects of contents during same state Encoding of faces versus houses 

Longitudinal study Test-retest of the same cognitive function 

• Learning effects 

• Aging effects 

• Disease effects (only for diseases that changes over time) 


Between-subject designs 

Examples 

Manipulation effects 

Training effects 

Patients and controls 

Different groups of patients 

Attention processing in ecstasy abusers versus non-users 

Effect of juggling training on motor cortex activation 

Working memory function in schizophrenic patients versus healthy controls 

Encoding in patients with Alzheimer's disease versus patients with semantic dementia 


designs , a comparison can be made between two different 
processing states, between different contents during 
a given state, or as effects of time. Between-subjects 
designs allow for studying group effects in a number 
of ways, including comparison of patients and 
healthy subjects, between different groups of patients, 
or between two groups who have been manipulated 
(e.g. trained) differently. A list of some possible designs 
is presented in Table A.l. In this way, it is imperative to 
pay attention to what comparisons are being made in a 
given experiment. 

When studying the effects of one condition, one 
must always contrast that to some other condition to 
achieve any meaningful data. Let us demonstrate this 
with a study on the perception of facial expressions. In 
a standard emotion task in neuroimaging (Del Ben et al., 
2005), subjects are asked to look at images of different 
faces. Half of the faces are female and the subject's task 
is to report whether the face is a male or a female. At 
the same time, the faces also vary according to what 
kind of emotion is expressed. The faces can either be 
neutral or aversive (e.g. sad or frightened). During 
the experiment, our main focus is on the difference 


between seeing aversive and neutral faces. During the 
analysis of the data, we first determine when the aver¬ 
sive and the neutral faces were shown. We then deter¬ 
mine the signal intensities for each of these periods and 
look at the mean signal intensity for neutral and aver¬ 
sive faces, respectively. The mean signal and distribu¬ 
tion in the brain for each condition are shown in Figure 
A.24. If we are interested in identifying areas that are 
involved in the processing of aversive faces, we take 
the mean activity during aversive stimuli and subtract 
the mean activity of all neutral stimuli. In this way, 
all activations that are common for both conditions, 
including primary visual processing and rudimentary 
face perception, are cancelled out. What we end up 
with is where in the brain we see specific rise (or fall) 
in activation when the subjects see aversive faces. 

BOLD fMRI is probably the most used measure of 
brain activation today. It has demonstrated its utility 
in every aspect of cognitive neuroscience and is con¬ 
tinuously being developed as a research tool. With 
increasing technological advances the method will 
lead to a better understanding of the temporal and 
spatial workings of the mind and brain. 
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FIGURE A.24 fMRI for aversive emotional faces. Looking at 
aversive faces activates emotion areas of the brain. In a series of dif¬ 
ferent face stimuli, some faces are aversive (e.g. frightened, sad, and 
showing repulsion) and others are neutral. If we subtract the activ¬ 
ity associated with neutral faces from aversive faces, we get neural 
signal that is selective for looking at aversive faces. Here, bilateral 
amygdala and orbitofrontal activation can be seen in an individual 
subject using this contrast. Source : Del Ben et al., 2005. 

2.4.2 Arterial spin labeling (ASL) 

By altering the magnetic properties of the blood flood¬ 
ing into the brain, an MRI measurement can also be 
made sensitive to the blood perfusion in the brain. 
A technique called arterial spin labeling uses this idea 
by 'labeling' the blood in the carotid artery by apply¬ 
ing a brief radiofrequency pulse. This alters the mag¬ 
netic properties of this part of the blood. As this blood 
continues to flood into the brain, it makes it possible 
to measure the relative change in magnetic suscep¬ 
tibility of a region of the brain, or the entire brain. 
A perfusion-weighted image can be performed by 
subtracting a baseline brain image with no magnet¬ 
ized blood from a brain image with magnetized blood. 
In this way, one can measure the blood perfusion in a 


region of the brain and it is also possible to compare 
perfusion images between groups; see Box A.5. 

2.4.3 Other specialized MRI sequences 

Diffusion tensor imaging (DTI) and tractography 
MRI can also be used to measure the movement of 
water molecules over time. In a free and unconstrained 
(isotropic) environment, water will diffuse equally in 
all directions, also known as Brownian movement. If 
you measure the diffusion in this medium, the resulting 
image will be a sphere. If water is put into a more con¬ 
strained (anisotropic) environment, however, it cannot 
diffuse freely but can only move along the structures 
that it is physically limited by. For example, in a glass 
of water, the water in the middle can move freely in all 
directions. The movement potential of one water mol¬ 
ecule is equal in all directions. However, if you put a 
drinking straw into the water, the movement potential 
of the water molecules within that tube is dramatically 
limited. If you now measure the water diffusion within 
this tube, it is no longer circular or equipotent, but an 
oblong sphere (Figure A.26). 

In biological matter such as the brain, the water 
diffusion is significantly limited. However, there is a 
systematic difference between gray and white mat¬ 
ter. While gray matter has little inner structure in the 
sense of limiting water diffusion, white matter consists 
of fibers that constrain the diffusion in some directions 
more than others. Just like putting one or more straws 
into the glass of water hinders water from moving 
through the straw walls, neuronal fibers constrain 
water diffusion across, but not along, the fiber direc¬ 
tion (see Figure A.26(b)). Connections between brain 
areas occur as bundles of fibers (axons). By using dif¬ 
fusion tensor imaging, it is possible to measure the 
relative direction and coherence of these white matter 
tracts. Thus, DTI can be used to measure white matter 
changes in neurological disease, but it is also possible 
to determine how a selected area is connected to other 
brain areas by following the fibers from the selected 
region, a method called fiber tracking. The use of DTI 
in a healthy brain can be seen in Box A.6. 

MR spectroscopy 

Magnetic resonance spectroscopy (MRS) is closely 
related to magnetic resonance imaging. Both tech¬ 
niques use the magnetic properties of atomic nuclei to 
get information about a biological sample. However, 
there is a crucial difference: while MRI measures the 
spatial distribution of magnetization, MRS measures 



Amygdala 



Orbitofrontal cortex 


















580 


APPENDIX: METHODS FOR OBSERVING THE LIVING BRAIN 


BOX A.5 ASL of Alzheimer’s disease and mild cognitive impairment 


Although significant cognitive changes can be detected 
at an early stage of Alzheimer's disease (AD), including 
memory deficits, more physiological changes also occur 
in the brain. By studying the perfusion of blood in these 
patients, it is possible to see what areas show a general 
lower metabolism or flow of blood. In such a study, 
Johnson et al. (2005) used arterial spin labeling to com¬ 
pare the perfusion levels in a group of AD patients to that 
of healthy old subjects and subjects suffering from mild 
cognitive impairment, a pre-diagnostic syndrome where 
at least a subgroup show a high conversion rate to AD. 

The researchers found a significant decrease in per¬ 
fusion in the AD group, compared to the control group, 
in a number of regions in the parietal cortex, posterior 
cingulate cortex, and frontal cortex. At a lower statisti¬ 
cal threshold, the MCI group demonstrated a lower per¬ 
fusion in the right inferior parietal lobe, the same place 
where the largest effect was seen in the AD group. This 
study demonstrates that perfusion scanning using ASL 
can be used to detect differences between groups at dif¬ 
ferent stages in Alzheimer's disease. 



FIGURE A.25 ASL in Alzheimer's disease. Source : Johnson 
et al., 2005. 
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FIGURE A.26 Diffusion of water depends on the local environ¬ 
ment. (a) In the free and unrestricted medium (i.e. a glass of water) water 
can diffuse freely. The diffusion is isotropic ; it has the potential to move 
in all directions. If the water molecule is physically restricted it can no 
longer move freely in any direction. This diffusion is anisotropic ; it cannot 
move in any direction. In a medium of fibers such as the brain's white 
matter (schematically shown in (b)) water molecules are highly restricted 
by the axonal fibers. In this way, it is possible to visualize the fiber tracts 
of the brain and furthermore to estimate the integrity (homogeneity) of 
white matter in a given region, (c) Such visualization produces the typi¬ 
cal colored DTI brain image that displays different trajectory trends in 
regional white matter. 


the amount of signal from each chemical environment 
(chemical or molecular distribution). In other words, 
MRI allows one to study a particular region within 
an organism or sample, but gives relatively little 


information about the chemical or physical nature 
of that region. MRS, which is an NMR method, pro¬ 
vides a wealth of chemical information about the 
same region. The frequency distribution is called a 
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BOX A.6 Reading ability and brain connectivity 


Reading is a complex cognitive skill that is the result of 
coordination between many different brain regions. In 
order for such a communication between brain regions 
to occur, there must be physical contacts between these 
areas. This can be either through direct connections 
between two or more areas, or indirect connections 
through one or more additional areas. Although many 
studies using functional imaging address the cooperation 
between such areas, they do not provide direct evidence 
about the structural connections that are present in the 
brain to support this cooperation. 

In a study by Beaulieu and colleagues (Beaulieu et al ., 
2005), this was addressed by studying the brain connec¬ 
tivity in children with diverse reading abilities. By com¬ 
paring white matter connectivity and integrity changes 
and reading ability in children, Beaulieu discovered five 
white matter areas that were good indicators of the chil¬ 
dren's reading ability. 


The researchers went on by looking at the different 
connectivities in these areas. By using each white mat¬ 
ter area as a 'seed point', the researchers could detect 
and visualize the connectivity of each cluster. These were 
generally ordered in three groups: one going between the 
front (anterior) and back (posterior) ends of the brain; 
one group of left-right oriented fibers; and one traveling 
from the upper (superior) to the lower (inferior) parts of 
the brain. 

In this way, Beaulieu and colleagues were able to 
determine both the white matter regions that are impor¬ 
tant in reading skills in children and, at the same time, 
say which pathways these areas represented. Following 
these discoveries, it is now possible to generate more 
specific hypotheses about the neural substrate for read¬ 
ing skill acquisition, how these areas could be function¬ 
ally connected, and to test them using other methods, 
e.g. fMRI. 




FIGURE A.27 Connectivity patterns in the brain were investigated by Beaulieu and colleagues (2005) using 
DTI. (a) Shows the brain areas (in purple) selected for investigation. Four were in the left hemisphere and one in the 
right, (b) Shows white matter tracts (in yellow) for the preselected regions that have an anterior-posterior (front to 
back of the brain) orientation, (c) Shows white matter tracts (in red) for the preselected regions that have a left-right 
orientation, (d) Shows white matter tracts (in blue) for the preselected regions that have a superior-inferior (top to 
bottom of the brain) orientation. Source : Beaulieu et al., 2005. 
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FIGURE A* 2 8 Spectroscopic MRI detects chemical differences 
in brain pathology. Left: (green), the healthy part of the brain shows 
normal NAA levels (green peak), while on the left-hand side this 
NAA level has diminished, while lactate has a significantly altered 
(red) drop. Source: Courtesy of Lars Hanson, Danish Research Centre 
for Magnetic Resonance, Copenhagen University Hospital, Denmark. 


spectrum and is analogous to the optical spectra of 
substances which are responsible for their visible 
colors. In this way, MRS can measure the chemical 
composition in a given region of the brain. MRI and 
MRS can be combined to provide the spatial distribu¬ 
tion of chemical compounds as shown in Figure A.28. 
This figure shows the spectrum from a region of the 
brain and the corresponding chemical distribution of 
this region. If you compare this healthy area with an 
area from a diseased brain region, you can see some 
dramatic changes. First, from the healthy hemisphere 
it is possible to see that the N-acetylaspartate (NAA) 
signal is the highest of all molecular compounds. The 
NAA is a marker of mature axonal white matter in the 
living brain (Bjartmar et al., 2002). By contrast, this is 
precisely the same signal that drops dramatically if 
we measure the same area in the lesioned hemisphere. 
Even without such dramatic effects, MRS can be used 
to study degeneration of brain tissue in diseases such 
as multiple sclerosis (De Stefano et al., 2005), stroke 
(Saunders, 2000), and Alzheimer's disease (Falini 
et al, 2005), but also the effects of substance abuse 
(Magalhaes, 2005; Reneman et al, 2006). 


2*5 MRI - a tool for the future 

With its superior spatial resolution and multiple uses, 
the MR scanner is an indispensable tool in cognitive 


neurosciences. The advances in MRI come from many 
directions but, in general, we can speak of two cate¬ 
gories of advancements: technical and analytical tools. 
Technical advances include the production of scanners 
operating at higher field strengths. Increased field 
strengths enhance the scanner's ability to record sig¬ 
nals. So, by exchanging a 1.5 Tesla (1.5 T) scanner with 
a 3 T scanner, we get a higher spatial resolution in both 
the structural images as well as the functional images. 
While 3 T or 4 T is the current high-field standard in 
scanning subjects and patients, scanners are already 
available at higher field strengths such as 7 T and 11T. 
Scanning the cortex at 7 T or higher field strengths 
has now demonstrated the possibility to make out the 
different layers of the cortex (Fatterpekar et al., 2002; 
Zhao et al., 2006). This is important not only because 
we get a higher resolution for studies already per¬ 
formed, but it will also generate whole new ways to 
study the brain, and our ideas about its workings. 

Advances in field strengths are complemented by 
other areas such as improvements in the apparatus 
that generate the magnetic pulse, or the receiver that 
records the signal. Such ongoing improvements are 
likely to make significant contributions to the possibil¬ 
ity of scanning the brain with increasingly higher res¬ 
olution and sometimes invent novel ways to acquire 
the data. While these advances promise a better reso¬ 
lution, they are also associated with specific problems. 
For example, the higher field strength that produces 
better signal-to-noise ratio in most parts of the brain 
leads to greater loss of signal in other areas. Since the 
BOLD signal (see previously) relies on the relative 
amount of oxygenated blood in an area, areas with 
oxygen that are not part of the brain influence the sig¬ 
nal in that area. The medial temporal lobe and ventral 
prefrontal cortex both lie close to the nasal air cavities. 
As a result the BOLD signal in these brain regions is 
corrupted by the non-brain areas filled with oxygen. 
When moving from field strengths of 1.5 T to 3 T, this 
became a problem that needed to be addressed. fMRI 
activation data from the hippocampus, amygdala, and 
orbitofrontal cortex were distorted by a loss of signal 
at 3 T. To overcome this problem, studies now use spe¬ 
cialized sequences that minimize the artifacts in these 
areas. However, when moving from the standard 3 T 
or 4 T to 7 T or higher field strengths, these problems 
re-emerge. Only new advancements in noise reduction 
methods can solve these problems. 

The other general area of advance is software 
improvements. In a few years, the field of neuroimag¬ 
ing has seen a significant expansion in the number of 
ways to analyze data. Many of these improvements 
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FIGURE A. 2 9 Increased field strength leads to higher resolution 
images. By comparing the image of the same subject on both a 1.5 T 
and a 3 T scanner it is possible to see the resolution increase. Here, the 
pathological changes occurring in sclerosis are best seen using 3 T (b 
and d) while several lesions are not even detected at 1.5 T (a and c, 
see arrow in d). All image acquisitions were done with the injection of 
a contrast agent, gadolinium. Source : Nielsen et al., 2006. 

are not isolated to MRI alone, but are relevant to most 
or all neuroimaging approaches. While the early days 
of fMRI neuroimaging analyzed trial blocks, later 
advances, such as the event-related paradigm, has 
led to new analytical tools. One such is the dynamic 
causal modeling (DCM) analysis of neural activation 
patterns. In general, DCM is a way to analyze how 
activations across the brain occur at the same time or 
are caused. There are two main ways to analyze such 
data. An analysis of the functional connectivity focuses 
on correlations between brain areas. For example, 
one can ask whether there is a contingent relationship 
between activation in the hippocampus and the dorso¬ 
lateral prefrontal cortex (DL-PFC) during an encoding 
task. Here, one can get an estimate of how correlated 
two (or more) areas of the brain are - in other words, 
the relative activation strength between two or more 
areas. The second kind of analysis is to look at the 
effective connectivity between areas. Here, one tries to 
move beyond mere correlation and estimate the rela¬ 
tive cause and effect relationship between areas. In 
our example of hippocampus and DL-PFC, we move 
beyond asking how correlated the regions are and now 


how one area is causing the other to activate. Today, 
both approaches are being developed continuously 
and are increasingly being used in the analysis of neu¬ 
roimaging data. However, since they rely on specific 
experimental design and data sampling, only para¬ 
digms specifically tailored to the analysis can be used. 

2*6 Optical imaging 

Optical imaging is a quite recent addition to the brain 
imaging toolbox. A laser source of near-infrared light 
is positioned on the scalp. A bundle of optical fibers is 
used as detectors and placed a few centimeters away 
from the light source. The detectors record how the 
path of light from the laser source is altered, either 
through absorption or scattering, as it traverses brain 
tissue. This information is used in two ways. First, it 
can measure the concentration of chemicals in the 
brain by measuring the absorption of light in an area. 
Second, it can provide information about more physi¬ 
ological properties of the brain that are associated to 
the level of neuronal firing. This is done by measur¬ 
ing the scattering of light, which is an indicator of 
swelling of glial cells and neurons. In this way, optical 
imaging provides a simultaneous measure of both the 
source and time course of neural activation within an 
area of the brain. 

In a study using optical imaging, Sato and col¬ 
leagues (Sato et al., 2005) studied the activity of som¬ 
atosensory cortex in the pre-surgical planning stage 
of patients with brain tumors or epilepsy. As with 
previous findings in the human and non-human ani¬ 
mal literature, Sato et al. found that the organization 
of the somatosensory cortex consisted of neighboring 
response areas, for example, the fingers, and that the 
areas demonstrated a certain amount of functional 
overlap. This is illustrated in Figure A.30. It is also 
noteworthy to consider the relation between these 
findings and the study of brain plasticity in phantom 
limb sensation presented in Box A.2. This study also 
demonstrates that optical imaging has a potential use 
in pre-surgical planning. 


3*0 MULTIMODAL BRAIN IMAGING 

Neuroscience today rests on a number of different imag¬ 
ing techniques (modalities) and we have only mentioned 
the most prominent here. Each approach continues to 
provide novel findings that contribute to our under¬ 
standing of the brain and our cognitive apparatus. But, as 
Figure A.l demonstrates, each imaging technique has its 
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FIGURE A.30 Optical imaging: The cortex responds visibly to stimulation. Intrinsic optical responses to stimulation of the digits from 
the left somatosensory cortex of a 47-year-old patient. The detected optical responses are illustrated in added color superimposed on a vascu¬ 
lar image. Source : Adapted from Sato et al., 2005. 


strengths and weaknesses, especially in terms of its rela¬ 
tive spatial and temporal resolution. An obvious solu¬ 
tion is therefore to compare findings from studies using 
these different imaging approaches. However, since 
each method is different from the rest, we will never get 
identical results. Instead, what we get is added informa¬ 
tion about our area of interest. For example, if our study 
focuses on subjects reading a text aloud, we can compare 
the results from EEG and fMRI. In this way, we can get a 
better understanding of both the localization of areas that 


are responsible for text reading (fMRI) and, at the same 
time, get results about the millisecond-to-millisecond 
changes in activation levels during the task (EEG). 

Imaging modalities can be compared in many ways. 
In addition to the comparison of the results in different 
studies, an often-used approach is to combine imag¬ 
ing techniques more directly. A PET study will most 
often co-register its findings to a standard MRI or CT 
structural brain. The same is actually done with fMRI 
images: the activation images are co-registered to the 
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FIGURE A.31 The brain substrate for delta power during non-REM sleep. The image shows regional changes 
in the cerebral blood flow as a function of delta power during non-REM sleep. The activations are shown on a mid- 
sagittal slice (left) and different axial slices (right). Source : Dang-Vu, 2005. 


structural scans. This process gives us the opportunity 
to have a better ability to see where in the brain our 
changes are found. The activation images in them¬ 
selves bear too little information about the localization 
of the activations to be meaningful. 


3*1 Simultaneous imaging from 
different sources 

While most comparisons are done on images that have 
been recorded separately, for example, the PET scan and 
the structural MRI scan, it is also possible to record and 
co-register results from recordings that have been done 
simultaneously. For example, Dang-Vu and colleagues 
(Dang-Vu et al., 2005) used combined EEG and PET to 
study the contributions of brain areas in different lev¬ 
els of sleep. The EEG and PET recordings were done 
simultaneously while the subjects were sleeping. The 
researchers then identified the unique stages of sleep 
from the EEG patterns and focused on delta activation 
during non-REM (NREM) sleep and compared this to 
REM sleep and normal wakefulness (see Figure A.31). 
The researchers found a negative correlation between 
delta power and regional cerebral blood flow in the 
ventromedial prefrontal cortex, the basal forebrain, the 
striatum, the anterior insular, and the precuneus. These 


findings thus hint about areas that vary in delta activa¬ 
tion according to our level of awareness. 

Similarly, Laufs and colleagues (Laufs et al., 2003) 
combined EEG and fMRI measurements in the study 
of subjects at rest. The study of Testing state' must be 
seen as controversial, due to the fact that neither the 
mind nor the brain can be seen as being at 'rest' at 
any time. Laufs et al. assessed the different patterns of 
activity occurring within phases in which subjects were 
'at rest' and found that the resting phase consisted of 
'intertwined yet dissociable dynamic brain processes' 
in the EEG. In other words, the neuroimaging studies 
imply that the brains of subjects at rest are doing dif¬ 
ferent things at different times within the rest period. 
Furthermore, the researchers were able to make out 
separate neural networks underlying EEG bands, such 
as the alpha and beta band activity. 

It should be noted that there is a substantive dif¬ 
ference between combining EEG with PET and with 
fMRI. While the PET scanner does not induce artifacts 
that are insurmountable for the EEG, the MR scan¬ 
ner produces artifacts in the EEG so substantial that 
attempts until only recently have been unsuccessful 
at such a combination. One approach has been to read 
the EEG from people between fMRI runs and, in this 
way, assess the state of awareness, e.g. whether they 
are sleeping or awake. Through proper noise filtering 
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it has now become possible to filter out the scanner 
artifacts and make the EEG data useable even from 
within the fMRI scanner. In this way, it has become 
possible to combine the high temporal resolution from 
the EEG with the high spatial resolution in the MRI. 
Still, the filtering of artifacts is seen as problematic - 
we may see a significant loss of true signal, while 
some residual artifacts (even from the filter applied) 
can occur. Combined measures such as the EEG/fMRI 
must therefore still be interpreted with caution. 

3.2 Imaging genetics 

Within the area of multimodal brain imaging, new 
important steps are taken to combine our understand¬ 
ing of how genes make up your mind. Genes act at the 
molecular level in the body and thus act as the very 
building blocks of neurons. In this way, individual 
differences in the genetic makeup can influence the 
workings of the brain in significant ways. 

A natural variation in the promoter region of the 
serotonin transporter gene (called 5-HTTLPR) has been 
linked to alterations in serotonin transcription as well 
as serotonin uptake. Individuals who have two copies 
of the long (1) variant have a higher concentration of 
the serotonin transporter mRNA and therefore have 
greater serotonin uptake in comparison to individuals 


who have two versions of the short (s) variant. These 
subjects have a relatively lower concentration of the 
transporter and, as an effect, relatively greater synap¬ 
tic serotonin levels. Hence, the genetic makeup you 
might have in this area influences how much or little 
serotonin you have available in the brain. Just looking 
at your genetic makeup with respect to the serotonin 
transporter gene, it is now possible to predict the level 
of active serotonin in any given person. 

By applying such a combination of neuroimaging 
and genotyping, Hariri and colleagues (Hariri et al., 
2002) demonstrated that the type of genetic makeup 
had implications for the response of the amygdala to 
pictures of facial expressions (Figure A.32). Subjects 
with the short genetic version demonstrated elevated 
levels of amygdala activation, presumably caused by 
a higher concentration of serotonin in these subjects. 
This genetically driven difference in amygdala excit¬ 
ability might contribute to the increased fear and anxi¬ 
ety typically associated with the short genetic version. 
As such, the study by Hariri and colleagues is a pow¬ 
erful demonstration about what can drive the individ¬ 
ual differences in how the brain works (Figure A.33). 
Today, many researchers hold the view that this area 
of 'imaging genetics' holds the promise to give sig¬ 
nificant new insights into the workings of the brain 
and mind. 



FIGURE A.32 Genes, anger, and the amygdala, (a) Self-reported anger scores (on a scale called ANGER) are higher in males with high 
levels of specific TT and CT base pair sequences of the DARPP-32 gene, compared to CC sequences in the same DNA regions of the same indi¬ 
viduals. (b) An MRI shows that in these subjects the gray matter volume (the neural cell bodies) of the left amygdala is significantly lower than 
in the control group. Source : Reuter et al., 2009. 
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FIGURE A.33 Multileveled understanding of brain and mind. Four levels of understanding the mind and its brain. Imaging 
genetics allows for the estimation of genetic effects at the level of brain information processing, which represents a more proximate 
biological link to genes as well as an obligatory intermediate of behavior. Source : Adapted from Hariri et al., 2006. 
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FIGURE A.3 4 (a) TMS. When researchers operate a TMS coil near a 

participant's scalp, a powerful (• 2T) and rapidly (• 300 ms) changing mag¬ 
netic field passes painlessly through skin and bone. Because the strength fo 
the magnetic field falls off very rapidly with distance from the TMS coil, it 
can penetrate only a few centimeters and this means that only superficial 
areas of the brain are most effectively stimulated. The induced electric field 
causes electric current in nearby neurons, thus stimulating targeted regions of 
the cortex, (b) tACS. This method relies on application of alternating currents 
through an electrode. Electrical currents are applied constantly at low inten¬ 
sities (1 mA) over period of time in the order of seconds, to achieve changes 
in cortical activity The waveform of the stimulation is sinusoidal and differ¬ 
ent frequencies can be used during stimulation (up to 250 Hz). Source : Thut & 
Miniussi, 2009. 


Transcranial magnetic stimulation (TMS) 

Based on the findings from different imaging meth¬ 
ods, one will have a better understanding of the 
relationship between regional brain activation and 
cognition. However, these studies most often suffer 
from being correlational: they often only imply a rela¬ 
tionship between activations and cognition. Imagine 
that in a given task, you find that three regions seem to 
play important roles. But in order to go beyond these 
findings, you may wish to study the effects of lesions 
to these regions. Patient lesions are always a valuable 


source for performing such tests, but lesions rarely 
follow tight anatomical boundaries or functional 
activations. Therefore, using a method that is revers¬ 
ible and that influences the gross operations of small 
regions of the brain is attractive. 

Transcranial magnetic stimulation (TMS) is one 
such method. In TMS, rapidly changing magnetic 
fields (electromagnetic induction) are used to induce 
weak electric currents in the brain (Figure A.34). 
Although the exact nature of the operations of TMS 
are not yet fully understood, researchers now typically 
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divide between two kinds of TMS: single/paired pulse 
TMS (s/pTMS) and repetitive TMS (rTMS). 

In s/pTMS, pulses cause populations of neurons 
in the neocortex to depolarize and discharge an action 
potential. If applied to the primary visual cortex, 
the excitation of this region will make the person expe¬ 
rience small flashes of lights (phosphenes). If applied 
to the motor representation of the hand, one can elicit 
hand movement, or measure muscle tightening in the 
hand. 

Similarly, s/pTMS can be used to grossly inhibit 
activation in a particular region of the neocortex. 
For example, by applying TMS pulses to the tempo¬ 
ral lobe region bilaterally, Overgaard and colleagues 
(2004) demonstrated changes in consciousness of vis¬ 
ual stimuli, but not in the ability to guess correctly the 
property of the stimuli. 

In this way, TMS can be used to induce temporary 
and reversible "lesions" in healthy individuals. This 
method can be used to temporarily disrupt the func¬ 
tion of a particular region of the brain. If this affects 
task performance it may be deduced that this part of 
the brain was involved with the task at that particular 
time point. 

The second TMS method, repetitive TMS, produces 
effects that last longer than the period of stimulation. 
By applying repeated sequences of TMS pulses, it is 
possible to grossly increase or decrease the excitability 
of corticospinal or corticocortical pathways. Although 
the mechanisms involved in rTMS are not fully under¬ 
stood, it has been discovered that the level of excit¬ 
ability depends on the intensity of stimulation, coil 
orientation, and frequency of stimulation. It has been 
suggested that rTMS works through changes in syn¬ 
aptic efficacy akin to long-term potentiation (LTP) and 
long-term depression (LTD) (Fitzgerald et al., 2006). 

In a recent study, Babiloni and colleagues (2007) 
demonstrated that rTMS to the ventral parietal cortex 
affected both attentional processes and subjects' sen¬ 
sory consciousness. This suggested that the ventral 
parietal cortex plays a functional role on visuospatial 
attention and primary consciousness. 

In recent years, studies have also discovered that 
rTMS applied to specific regions may lead to neuro¬ 
physiological changes. For example, studies have 
demonstrated that rTMS to the prefrontal cortex 
induces changes in dopamine levels of the striatum 
and caudate nucleus (Strafella et al, 2001, 2003), pos¬ 
sibly by acting through corticostriatal networks. 

Through its diverse uses, TMS is today a method 
that is still gaining in popularity with scientists. In 
addition, the method has demonstrated some promise 


in the treatment of depression and other mood disor¬ 
ders and in Parkinson's disease. 


4.0 CONCLUDING REMARKS 

Neuroimaging has grown into one of the most impor¬ 
tant approaches to our increased understanding of 
the human brain and mind. Advances within each 
modality - be it optical imaging, EEG, or fMRI - rapidly 
change ways in which data can be analyzed and even 
the way we are posing our questions. Continuing 
innovations, both within hardware and software, 
make any prediction of the future of neuroimaging 
hazardous. However, some trends are very likely to 
continue, including the continuing increase in higher 
resolution in all modalities and new ways of combin¬ 
ing data from different modalities. 

We must keep in mind what we are really study¬ 
ing. The study of the human mind and its brain is 
founded on the very building biological blocks of 
our body: our genes. Our genes determine how and 
when our brain develops and it is thus the recipe for 
our mind that we should seek to understand. In order 
to do this we need to understand the levels at which 
we are working to understand ourselves. Here, let us 
distinguish between four levels. The first level is the 
genetic and how we understand how genes make up 
the recipe for cellular components such as proteins 
and transmitters. Second, we must understand how 
single neurons work and how they collaborate in net¬ 
works. Third, we can study the brain to see morpho¬ 
logical and functional factors influencing thought. 
Finally, we can study how effects at any of the pre¬ 
vious levels have an effect on our thought, behavior, 
and even personality. 
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Glossary 


acetylcholine (Ach; uh-see-tel-KO-leen): A neurotransmitter 
in the central and peripheral nervous system. See Chapter 16. 

acetylcholinesterase (Ache; uh-see-tel-KO-leen-ES-ter-ayze): 
An enzyme that deactivates acetylcholine, to clean up the 
synapse. 

acoustical analysis (uh-KOO-sti-kul uh-NAL-ih-sus; Greek 
akoustikos = of hearing): The process of interpreting physical 
sound energy, whether linguistic, musical, or sounds in the 
environment (as in a door slamming or a car starting). See 
Chapter 11, Introduction'. 

action potential (AK-shun po-TEN-shul): In neurons, an elec¬ 
trochemical signal beginning near the cell body and traveling 
down the axon to the synaptic terminal. Also called a 'spike' 
or 'neuronal firing'. See Chapter 3, 'Introduction'. 

adenosine (uh-DEN-oh-seen): An inhibitory neurotransmit¬ 
ter that is believed to promote sleep and suppress waking 
arousal. See Chapter 16. 

adrenaline (uh-DREN-uh-lin; from Latin, ad- = on top of; 
renes = kidney; -in = protein): Also called epinephrine, it is a 
peripheral hormone and brain neurotransmitter. As a circula¬ 
tory hormone it is released by the adrenal gland, located on 
top of the kidneys, and prepares the body for fight or flight. 
See Chapter 16. 

affective blindsight (uh-FEK-tiv BLIND-site; Latin affectus = 
emotion): An inability to understand emotional facial expres¬ 
sions. See Chapter 13, 'The Fear System'. 

agonist (AGG-uh-ist; Latin for competitor). A molecule that 
binds to a neural receptor, thereby increasing the chance 
that the postsynaptic neuron will fire. See Chapter 16, 
'Remembering the Words'. 

alpha waves (AL-fa WAY-vz; first letter of Greek alphabet): A 
regular electromagnetic wave detected in the brain or on the 
scalp, and apparently reflecting the activity of large popu¬ 
lations of neurons. Alpha waves have a frequency of 7.5 to 
13 Hz and originate predominantly from the occipital lobe 


during periods of waking relaxation with the eyes closed. 
Conversely, alpha waves are decreased when the eyes are 
open, as well as by drowsiness and sleep. See Chapter 4, 'A 
Range of Useful Tools'. 

amino acid (uh-MEE-no AE-sid): A constituent part of 
a protein molecule. Amino acids are critical to life, and have 
a large variety of metabolic roles. All amino acids contain 
both an NH 3 (amine) and a COOH (carboxyl) group. See 
Chapter 16. 

amnesia (am-NEE-zhuh; from Greek a-mn-sia = not mem¬ 
ory): A loss of memory. Two types are anterograde (a loss 
of memory after the time of the brain injury) and retrograde 
(a loss of memory before the time of the brain injury). See 
Chapter 9, 'Amnesia'. 

amygdala (uh-MIG-da-la; from amygdale = almond): The 
amygdalas are two small, almond-shaped masses of neurons 
located inside the tips of the temporal lobes. They are consid¬ 
ered part of the limbic system and play major roles in emo¬ 
tions like fear and trust, as well as in learning. See Chapter 1, 
'Some History, and Ongoing Debates'. 

antagonist (ann-TAG-uh-nist; Latin for competitor): A mol¬ 
ecule that binds to a receptor and blocks or inhibits its activ¬ 
ity. See agonist. 

anterior (ann-TEER-ee-er; from ante = in front of): Located in 
front of something. See Chapter 1 and the Mini-Atlas on the 
inside front cover. 

anterior cingulate cortex (an-TEER-ee-er SIN-gyu-lut COR- 
teks; from Latin ante = before, in front of; Latin cingulum = 
girdle; Latin cortex = bark): The frontal part of the cingulate 
cortex. The anterior cingulate cortex is involved in executive 
functioning. See Chapter 2, 'The Central Executive'. 

anterior commissure (an-TEER-ee-er KA-mih-shur; from 
Latin ante = before, in front of): A large bundle of nerve 
fibers connecting the two cerebral hemispheres. See Chapter 
5, 'Introduction'. 
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anterograde amnesia (AN-teh-ro-grayd am-NEE-zhuh; from 
New Latin antero- = forward; Greek a-mnesia = not memory): 
A form of amnesia in which events after the brain injury are not 
encoded in long-term memory, although events may be recalled 
from the period before the injury See Chapter 9, 'Amnesia'. See 
retrograde amnesia. 

aphasia (AY-PHAY-zha; from, a = without; Latin phasia = 
speech): A loss of language function due to brain injury, 
such as damage to Broca's area, for speech production, or 
Wernicke's area, for speech understanding. See Chapter 1, 
'Some History, and Ongoing Debates', and Chapter 7, 'Speech 
Perception'. 

apparent motion (uh-PAR-ent MO-shun; from Latin 
apparere = to appear; Latin movere = to move): A form of 
perceptual filling-in in which the brain creates the illusion 
of motion when a series of still images are flashed one after 
the other. See Chapter 6, 'Linking Brain Activity and Visual 
Experience'. 

arcuate fasciculus (AR-cue-ate fa-SIK-u-lus; Latin for curved 
bundle): A bundle of axonal fibers, especially the ones con¬ 
necting Broca's and Wernicke's areas in the left hemisphere. 
See Chapter 1, 'Some History, and Ongoing Debates'. 

area MT: A part of visual cortex that represents visual 
motion. See Chapter 6, 'Lunctional Organization of the Visual 
System'. 

array (uh-RAY): 1. A two-dimensional grid of sensory recep¬ 
tors, such as the retina. 2. A two-dimensional grid of higher- 
level elements in the visual hierarchy. Many 'layered' brain 
structures can be considered to be such arrays. See Chapter 3, 
'Arrays and Maps'. 

artificial neural net (ar-ti-LI-shel NOOR-el NET; from Greek 
neuron = nerve): Also known as ANNs or neural models, 
artificial neural nets are simulated, simplified models of 
brain functions. Most are relatively small in scale. However, 
they are important for understanding the principles of neural 
computation. See Chapter 3, 'Introduction'. 

associative process (uh-SO-see-a-tiv PRA-ses; from Latin 
ad- + sociare = to join): A process in which one or more sen¬ 
sory and/or response events are linked in the brain. See 
Chapter 5, 'Lrom 'Where' to 'What': The Lunctional Roles of 
Brain Regions'. 

attention (a-TEN-shun; from Latin attendere = to stretch 
out): Selection of some sensory, cognitive, or motor events 
to the exclusion of others. Attention is often taken to involve 
a focus on certain conscious events. Also see selective atten¬ 
tion. See Chapter 8. 

attention deficit (hyperactivity) disorder (ADD or ADHD; 

uh-TEN-shun DEL-ih-sit hy-per-ak-TIV-ih-tee dis-or-der): A 
persistent pattern of impulsiveness and difficulty with sus¬ 
tained attention, with or without a component of hyperac¬ 
tivity. There is some debate whether it is a genuine disorder 
or a normal adaptation that may be problematic today. It 
affects 3 to 5% of children around the world and continues 


into adulthood for 30-50% of those affected. See Chapter 12, 
'ADHD'. 

attention network task (ANT ; a-TEN-shun NET-werk TASK): 
A generalization of the flanker task, a tool for studying visual 
attention. The ANT allows testing of three separate aspects of 
attention: alerting before an expected signal, orienting to a 
specific location in space where the target is expected, and 
executive attention to act against expectations set up by the 
task. See Chapter 8. 

auditory cortex (AW-di-tor-ee kor-teks; from Latin 
auditorius = pertaining to one who hears; Latin cortex = bark): 
The parts of the cerebral cortex involved in processing sounds, 
such as Wernicke's area and Heschl's gyrus. See Chapter 7, 
'The Central Auditory System'. 

auditory scene analysis (AW-di-tor-ee SEEN uh-NAL-ih- 
sus): The process by which the auditory system segments 
and organizes the listening environment. See Chapter 7, 
'Introduction'. 

autism spectrum disorders (ASD; AW-tizm SPEK-trum dis- 
OR-der): A range of conditions characterized by deficits in 
social perception and communication, repetitive behavior, 
and other symptoms. 

autonomic nervous system (ANS; aw-to-NOM-ic NER-vus 
SIS-tern; from Greek neuron = nerve): The division of the 
peripheral nervous system that acts to maintain homeosta¬ 
sis and to regulate rest and activity. Physiological activities 
controlled by the ANS such as blood pressure and sweating 
are generally unconscious and nonvoluntary. See Chapter 5, 
'From 'Where' to 'What': The Functional Roles of Brain 
Regions'. 

automatic process (au-to-MA-tic PRAH-ses): A highly prac¬ 
ticed skill or habit that can be performed with minimal 
conscious involvement and voluntary effort. See Chapter 3, 
'Working Assumptions'. 

axon (AK-son): A long, slender branch of a nerve cell 
(neuron) that conducts electrical impulses away from the cell 
body. See Chapter 1, 'Some Starting Points', and Chapter 3, 
'Introduction'. 

Balint's syndrome (BA-lint-s SIN-drome): A brain disorder 
that is marked by simultanagnosia, the inability to perceive 
two or more objects at the same time. It may also include an 
inability to point to a target, and to shift gaze voluntarily. 
See Chapter 6, 'Brain Areas Necessary for Visual Awareness: 
Lesion Studies'. 

Baron-Cohen, Simon (b. 1958): Autism researcher who 
proposed that young children develop a theory of mind 
capacity, composed of four skills: detection of intentions of 
others, detection of eye-direction, shared attention with oth¬ 
ers, and implicit knowledge about others. See Chapter 14, 
'Overview'. 

basal ganglia (BAY-zel GAN-glee-uh; from Latin basis = step, 
base; Latin for swelling or excrescence): A large cluster of sub¬ 
cortical structures just outside of each thalamus, involving 
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motor control, automaticity, cognition, emotions, and learn¬ 
ing. See Chapter 5, 'Growing a Brain from the Bottom Up'. 

behaviorism (bi-HAY-vyer-ism): A philosophy of psychol¬ 
ogy proposing that all things organisms do, including acting, 
thinking, and feeling, should be regarded as behaviors. See 
Chapter 1, The Return of Consciousness in the Sciences'. 

beta waves (BAY-tuh WAYVZ; second letter of Greek alpha¬ 
bet): A band of irregular electromagnetic waveforms detected 
in the brain or on the scalp, and apparently reflecting the 
activity of large populations of neurons. The beta band has a 
frequency above 12 Hz and is associated with normal waking 
consciousness. Low-amplitude beta waves with multiple and 
varying frequencies often are associated with active, busy, or 
anxious thinking and active concentration. See Chapter 4, 
'A Range of Useful Tools'. 

binocular disparity (bih-NOC-u-ler dis-PAR-eh-tee; from 
Latin hi- = two; Latin oculus = eye; Latin disparare = to sepa¬ 
rate): The difference in perceived location of an object seen 
by the left and right eyes, resulting from the eyes' horizon¬ 
tal separation. The brain uses binocular disparity to obtain 
depth information from the retinal image in both eyes. See 
Chapter 6, 'Functional Organization of the Visual System'. 

binocular rivalry (bih-NOC-u-ler RYE-vel-ree; from Latin 
hi- = two; Latin oculus = eye): The alternating perception 
that occurs when a different pattern is shown to each eye 
and the brain cannot fuse them into a single, coherent per¬ 
cept. Instead, awareness of each eye's input appears and 
disappears for a few seconds. See Chapter 6, 'Linking Brain 
Activity and Visual Experience', Chapter 8. See bistable 
perception. 

bistable perception (BYE-STAY-bel per-SEP-shun; from 
Latin hi- = two): Sensory events that alternate between two 
perceptual interpretations. See Chapter 6, 'Linking Brain 
Activity and Visual Experience'. 

blindsight (BLIND-site): A type of brain damage in which 
patients can report some visual events with no subjective 
sense of seeing them, due to impairment of the first cortical 
area of the visual system, area VI. See Chapter 6, 'Brain Areas 
Necessary for Visual Awareness: Lesion Studies'. 

blood-brain barrier: A barrier between the bloodstream and 
the tissue of the brain, created by cells that line blood ves¬ 
sels in the brain. It allows the passage of small molecules like 
oxygen and glucose, while blocking larger molecules. See 
Chapter 16. 

blood-oxygen-level-dependent (BOLD) activity (BLUD 
OKS-eh-gen LEV-el dee-PEN-dent ak-TI-vi-tee): A magneti¬ 
cally induced physical signal that reflects the flow of oxygen 
in specific regions of the brain. The BOLD signal is the physi¬ 
cal source for functional magnetic resonance imaging. See 
Chapter 4. 

brainstem (BRAYN-stem): The lower part of the brain, con¬ 
necting to the spinal cord. All major motor and sensory 
systems pass through it, including the optic and auditory 
nerves. The brainstem also regulates cardiac and respiratory 
functions, and maintains conscious waking, slow-wave sleep 


(SWS), and REM dreams. See Chapter 5, 'Growing a Brain 
from the Bottom Up'. 

Broadbent, Donald (1926-1993): British cognitive psycholo¬ 
gist who developed the influential concepts of selective 
attention and working memory. See Chapter 2, 'Limited and 
Large-Capacity Functions'. 

Broca, Pierre-Paul (1824-1880): A French surgeon who stud¬ 
ied a brain-damaged patient with expressive aphasia, the 
inability to speak, while being able to understand speech. 
After the patient's death he was able to conduct a post-mor¬ 
tem identifying the damaged region as the left inferior frontal 
gyrus, now called Broca's area. See Chapter 1, 'Some History, 
and Ongoing Debates'. 

Broca's area (BRO-kas AIR-ee-a): The left inferior fron¬ 
tal gyrus, or its posterior segment, reported by Pierre-Paul 
Broca in 1861 to be responsible for the deficit of a patient who 
could not speak, but had preserved speech understanding. 
Other functions have since been attributed to Broca's area. 
See Chapter 1, 'Some History, and Ongoing Debates', and 
Chapter 7, 'Speech Perception'. 

Brodmann's areas (BROD-mans AIR-ee-uh): About 100 cor¬ 
tical regions defined and numbered by German neurologist 
Korbinian Brodmann, originally based on the microscopic 
anatomy of neurons in different patches of the cortex. They 
are still widely used for cortical localization, and Brodmann's 
areas generally have distinctive functions. See Chapter 5, 
'Introduction'. 

Buber, Martin (1878-1965): An Austrian-born Jewish phi¬ 
losopher best known for his philosophy of 'I and Thou' 
dialogue, emphasizing the relationships between conscious 
persons rather than reducing others to objects. See Chapter 
14, 'Overview'. 

cell assemblies (SEL uh-SEM-blees): Also called Hebbian 
cell assemblies, these are active networks of related neurons 
representing some sensory input or similar event. According 
to Donald O. Hebb's 1949 hypothesis, 'neurons that fire 
together, wire together', so that simultaneous firing causes 
the synaptic links in a cell assembly to grow stronger. See 
Hebbian learning. See Chapter 3, 'Working Assumptions'. 

Central Dogma of Molecular Biology, The: The hypothesis 
that genetic information flows in one direction only, from 
DNA to messenger RNA, and then to active proteins. The 
Central Dogma was first enunciated by Francis Crick in 1958, 
and has been modified by later discoveries showing numerous 
causal loops that operate in the reverse direction. However, 
DNA is still a tightly protected molecule that is rarely altered. 
However, its expression is constantly regulated - enabled or 
blocked by epigenetic events. See Chapters 15 and 16. 

central executive (CEN-trel eks-EK-yoo-tiv; from Latin 
centrum = center): Brain processes for planning, decision 
making, abstract thinking, rule acquisition, initiating and 
inhibiting actions, resolving goal conflicts, and flexible con¬ 
trol of attention. These functions relate to working memory 
and tend to involve the frontal lobes. See Chapter 2, 'Classical 
Working Memory' and 'The Central Executive'. 
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central nervous system (CNS; SEN-trel NER-vus SIS-tem; 
from Latin centrum = center; Latin nervus = sinew, nerve): 
The brain and spinal cord. All neurons outside of the CNS 
are considered to be the peripheral nervous system (PNS). 
See Chapter 5, 'Introduction'. 

central sulcus (SEN-tral SUL-cus; from Latin sulcus = 
groove): Also called the central fissure, this fold in the cer¬ 
ebral cortex is a prominent landmark of the brain that sepa¬ 
rates the parietal lobe from the frontal lobe and the primary 
motor cortex from the primary somatosensory cortex. The 
central sulcus is a clear dividing line between the input- and 
output-related areas of cortex. See Chapter 1, 'Some Starting 
Points'. 

cerebellum (ser-e-BEL-em; from the Latin word cerebrum = 
brain, cerebellum means 'little brain'): A major region of the 
brain located just below and to the rear of the occipital lobe of 
the cerebral cortex. The cerebellum plays an important role in 
the integration of sensory perception, fine motor control, and 
sensorimotor coordination. Recent evidence shows cognitive 
involvement as well. See Chapter 1, 'Some Starting Points', 
and Chapter 5, 'Introduction'. 

cerebral cortex (suh-REE-bral KOR-teks; from Latin 
cerebrum = brain; Greek cortex = bark): The outer surface 
of the great cerebrum, the largest part of the human brain, 
divided into two symmetrical cerebral hemispheres. Most of 
the cortex has six distinctive cellular layers, containing cell 
bodies with a gray appearance. But its long-distance nerve 
cells send out axons to other parts of cortex, to thalamus, and 
to other brain regions, which become covered with white 
supportive cells (myelin). As a result, a vertical cut of the cor¬ 
tex appears to the naked eye to have a thin, gray outer layer 
and a white inner mass, called the 'gray matter' and 'white 
matter', respectively. The cerebral cortex plays a key role 
in sensory analysis, spatial location, speech perception and 
production, memory, attention, emotion, motivation, action 
planning, voluntary control, thought, executive functions, 
and consciousness. See Chapter 1, 'Some Starting Points', 
and Chapter 5, 'Growing a Brain from the Bottom Up'. 

cerebrospinal fluid (CSF) (suh-ree-bro-SPEYE-nel floo-id; 
Latin cerebrum = brain): The internal circulation of the spine 
and brain. CSL allows for a protected flow of molecules and 
cells that is not exposed to the blood stream. 

cerebrum (suh-REE-brum; Latin for brain): See cerebral 
cortex. 

chromatin (KRO-muh-tin; Greek chroma = color): The 
substance of the chromosomes of each cell, including DNA, 
some RNA, proteins, and regulatory molecules that change 
the expression of DNA. 

chunking: A way to make efficient use of short-term mem¬ 
ory limitations by condensing large amounts of knowledge 
into a small symbolic units, rules, or regularities, called 
'chunks'. In natural language nouns can be considered to 
be chunks, since they allow us to refer to large bodies of 


knowledge by single words. See Chapter 2, 'Limited and 
Large-Capacity Lunctions'. 

Churchland, Patricia (b. 1943): Canadian-American phi¬ 
losopher of mind who won a Mac Arthur prize in 1991. 
Churchland's 'neurophilosophy' argues that popular con¬ 
cepts of mind will need to be revised as we learn more about 
the underlying brain functions. See Chapter 1, Box 1.2. 

cingulate cortex (SIN-gyu-lut KOR-teks; from Latin cing¬ 
ulum = belt; Latin cortex = bark): A part of the cortex on the 
medial (inner) surface of each hemisphere. It is involved in 
executive functions, the resolution of conflicting goals, and 
emotion. 

classical conditioning (KLASS-ih-kel con-DI-shun-ing): 
Also called Pavlovian conditioning. A form of associative 
learning in which humans or animals learn that an arbitrary 
stimulus (such as a bell) signals the coming of a biological 
stimulus (such as food). I. P. Pavlov called these the condi¬ 
tioned stimulus (CS) and unconditioned stimulus (US). If 
the CS is repeatedly presented just before the US, an animal 
begins to produce a behavioral response to the CS. Pavlov's 
proposal that conditional reflexes are the basic unit of all 
human learning is no longer generally believed. However, 
classical conditioning is widely used in research and is 
thought to be relevant to learned anxiety disorders and food 
preferences. See Chapter 13, 'Introduction'. 

cognitive neuroscience (KOG-ni-tiv NOOR-o-SI-ens; from 
Latin cognoscere = to know): An emerging integration of two 
previously separate fields of science, cognitive psychology 
and neuroscience. Most research in cognitive neuroscience 
makes use of psychological methods simultaneous with brain 
activity recording. See Chapter 1, 'Introduction'. 

cognitive set (KOG-ni-tiv SET; from Latin cogn = to know): 
A state of mental preparedness for some event or action. See 
Chapter 12, 'A Closer Look at Lrontal Lobe Lunction'. 

computed tomography (kom-PYOO-ted tom-OG-reh-fee; 
from Latin computare = to consider; Greek tomos = slice; 
Greek graphein = writing): Abbreviated as CT. Physiological 
recordings in which a three-dimensional image of a body 
structure (such as the brain) is constructed by computer from 
a series of slice images. See Chapter 4, 'Introduction'. 

confabulation (kon-fab-yoo-LAY-shen; from Latin fabula = 
story): A neurological symptom in which false memories 
or perceptions are reported with no intention to lie. See 
Chapter 9, 'Memories Are Made of This'. 

connectionism (keh-NEK-shun-ism; from Latin com- + 
nectere = to bind together): The study of artificial or biologi¬ 
cally based neural networks. See Chapter 3, 'How Neural 
Arrays Adapt and Learn'. 

consciousness (KON-shes-ness; from Latin con = together; 
scientia = knowledge): Awareness, wakefulness. Consci¬ 
ousness implies being sensitive and responsive to the envi¬ 
ronment, in contrast to being asleep or in coma. Synonyms 
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include explicit cognition and focal attention. See Chapter 1, 
'Some History, and Ongoing Debates' and 'The Return of 
Consciousness in the Sciences'; and Chapter 8. 

consolidation hypothesis (kon-SOL-ih-DAY-shun high- 
POTH-uh-sis): The process by which new memories are 
transformed into long-term memory traces. Memories may 
be stored in the same areas of the brain that support active 
moment-to-moment functions like perception and speech 
control. Consolidation may involve synaptic changes in such 
brain regions, which make active neuronal connections more 
efficient. See Chapter 2, 'Consolidation of Short-Term Events 
into Long-Term Memory'. 

contralateral (KON-tra-LAT-er-el; Latin for 'against the 
side'): The opposite side of the body or brain. See Chapter 2, 
'Action'. See ipsilateral. 

coronal (keh-RONE-el; from Latin corona = crown): A 
crown-shaped vertical slice of the brain that divides it into 
anterior and posterior halves. See Chapter 1, 'Some Starting 
Points'. 

corpus callosum (KOR-pus kal-OS-um; from Latin cor¬ 
pus = body; Latin callosum = tough): A massive fiber bridge 
between the right and left hemispheres, consisting of more 
than 100 neuronal axons. It appears white when cut, because 
the axons are covered by white myelin cells. See Chapter 1, 
'Some Starting Points'. 

cortex (KOR-teks): See cerebral cortex. 

cortical color blindness (KOR-ti-kel KUH-ler BLIND-ness): 
An inability to identify colors caused by damage to color-sen¬ 
sitive regions of the visual cortex. See Chapter 6, 'Brain Areas 
Necessary for Visual Awareness: Lesion Studies'. 

cortical column (KOR-ti-kel KAW-lum; from Greek cort = 
skin, husk; Latin columna = top): A barrel-shaped slice of 
the six surface layers of cortex that often contain closely 
related neurons. Columns are about 0.5 mm in diameter and 
2.5 mm in depth. They may be clustered into hypercolumns, 
which may be part of even larger clusters. See Chapter 5, 
'Introduction'. 

creole (KRAY-ole): A true language that children spontane¬ 
ously evolve in multilingual communities. Creoles often are 
encountered in island communities where language com¬ 
munities overlap. 'Creoles' are contrasted with 'pidgins', 
which are dialects typically spoken by adults as a second 
language. Creoles are remarkable because they exhibit a 
full-fledged grammar, unlike pidgins. The spontaneity with 
which they arise suggests that human infants and children 
may be equipped with a biological language capacity with 
universal features. See Chapter 11, 'The Nature of Language' 
and Box 11.2. 

Darwin, Charles (1809-1882): English naturalist who 
was among the first in modern times to make a persuasive 
case that all living species emerged from common ances¬ 
tors over very long stretches of time, by 'survival of the fit¬ 
test' and the most reproductively successful. Darwin also 


published important observations about emotional expres¬ 
sions in humans and animals. See Chapter 1, 'Some History, 
and Ongoing Debates'. 

declarative memory (deh-KLAR-a-tiv MEM-ree; from 
Latin declarare = to make visible): The capacity to recall 
facts and beliefs. A kind of explicit memory. See Chapter 2, 
'Classical Working Memory'. 

deja vu (DAY-zha VOO; French for 'already seen'): A feel¬ 
ing that one has lived through the present moment before. 
See Chapter 9, 'Varieties of Memory' and Box 9.3. 

delta waves (DEL-tuh WAYVZ; fourth letter of Greek 
alphabet): A band of slow, high-amplitude electromagnetic 
waveforms associated with deep sleep, and recorded in the 
brain or on the scalp, apparently reflecting large popula¬ 
tions of neurons. Delta generally is considered to be less than 
2.5 Hz. It coexists with waking EEG as well, but becomes 
visible in the raw (unprocessed) EEG only when delta 
predominates in sleep and drowsy states. See Chapter 4, 
'A Range of Useful Tools'. 

dendrite (DEN-drite; from Greek dendron = tree): One of 
numerous thin, branched micron-level tubes extending from 
the cell body of a neuron. Dendrites typically receive synap¬ 
tic stimulation from other neurons, and therefore serve as the 
input branches of the neuron. See Chapter 3, 'Introduction'. 

Descartes, Rene (1596-1650): A French philosopher, math¬ 
ematician, scientist, and writer who spent most of his adult 
life in the Dutch Republic. Descartes has been dubbed the 
'Father of Modern Philosophy' and was also a careful student 
of the brain. He often is considered the originator of modern 
mind/body philosophy. See Chapter 1, 'Some History, and 
Ongoing Debates'. 

developmental cognitive neuroscience (deh-vel-op-MEN- 
tel COG-ni-tiv NUR-o-si-ens): The study of the normal 
growth of the brain and its mental capacities. See Chapter 15, 
'Introduction'. 

diencephalons (die-en-SEF-a-lon; from Greek dia- = 
through; enkephalos = brain): The part of the brain that con¬ 
tains the thalamus, hypothalamus, and the posterior half of 

the pituitary gland. 

diffusion tractography (di-FYOO-zhen trak-TOH-greh-fee; 
from Latin diffusus = scatter; traherc = to pull; Greek graph- 
ein = writing): A brain imaging technique that tracks the dif¬ 
fusion of water molecules in order to trace the major neuronal 
pathways of the brain. See Chapter 4 and the Appendix. 

discourse (DIS-kors): A style of language usage in a com¬ 
munity, often taken to reflect a particular political slant. See 
Chapter 11, 'Introduction'. 

domain specificity (do-MANE spes-i-FIS-ih-tee): Func¬ 
tional specificity of brain regions or mechanisms. The idea 
that each cognitive function may have its own region or 
network of brain regions, rather than general-purpose brain 
mechanisms with multiple cognitive functions. See Chapter 
12, 'A Closer Look at Frontal Lobe Function'. 
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dopamine (DOH-puh-meen): A major neuromodulator and 
neurotransmitter that is produced in several different parts 
of the brain, including the substantia nigra and ventral teg¬ 
mental area. Dopamine has numerous functions, including 
voluntary movement control, reward, and inhibition of lacta¬ 
tion (milk production) in females, sleep, mood, attention, and 
learning. 

dorsal (DOR-sel; from dorsum = back): The upper part of a 
brain structure, also called superior. 

dorsolateral syndrome (dor-so-LAT-er-el SIN-drome; from 
Latin dorsum = back; latus = side; frons = forehead): Deficits 
caused by damage to the dorsolateral prefrontal cortex, such 
as 'flat affect' (lack of emotion) and an impaired ability to 
switch to and initiate new actions. See Chapter 12, 'Frontal 
Lobe Dysfunction'. 

dorsolateral prefrontal cortex (DOR-so-LAT-er-el pree- 
FRON-tal KOR-teks; from Latin dorsum = back; latus = side; 
pre = in front of; frons = the forehead; Greek cortex = bark): 
Prefrontal region involved in motor planning, executive 
control, self-regulation, emotion, and working memory. See 
Chapter 2, 'The Central Executive'. 

dynamic causal modeling (die-NAM-ic KOS-el MO-del-ing; 
from Greek dynamikos = powerful; Latin causa = cause; Latin 
modulus = small measure): A method for interpreting brain 
data, such as functional magnetic resonance imaging (fMRI), 
that helps to interpret causal relationships among brain activi¬ 
ties during a specified task. See Chapter 4, 'Correlation and 
Causation'. 

echolalia (eh-ko-LAY-lee-eh; Greek echo = to repeat; 
lalid = babbling): Constant and uncontrollable imitation of 
the speech of others. Echolalia may be seen in autism, Tourette 
syndrome, aphasia, developmental disability, schizophrenia, 
Asperger syndrome, Alzheimer's disease, and other condi¬ 
tions. See Chapter 12, 'Frontal Lobe Dysfunction'. 

echopraxia (eh-ko-PRAK-see-eh; from Greek echo = to 
repeat; praxis = action): A neurological symptom involving 
repeated, involuntary imitation of the movements of another 
person. See Chapter 12, 'Frontal Lobe Dysfunction'. 

Edelman, Gerald M. (b. 1929). American immunologist 
and neurobiologist who won the Nobel Prize for his work on 
the structure of antibody molecules. Edelman developed the 
theoretical framework of Neural Darwinism, which applies 
Darwinian selectionist principles to the brain, in contrast to 
the instructionist principles of conventional computers. See 
Chapter 3 and Chapter 16. 

electroencephalography (EEG; eh-LEK-tro-en-sef-eh-LOG- 
reh-fee; from Greek elektron = sunlight; en- + kephale = in the 
head; graphein = writing): Electrical activity that typically is 
recorded on the scalp and sometimes on the surface of the 
cortex, reflecting the electromagnetic field of large numbers 
of active neurons. See Chapter 4, 'A Range of Useful Tools', 
and the appendix. 


empathy (EM-path-ee; from Greek empatheia = passion): 
The capability to share one's feelings and understand another 
person's. See Chapter 14, 'Overview'. 

enzyme (EN-zime; Greek, en- + zyme = yeast): Molecules 
that facilitate the rate of chemical reactions. 

epigenesis (ep-ih-GEN-eh-sis; from Greek epi = after; 
Greek genesis = birth, origin): Non-DNA factors that shape 
cells during gestation (pregnancy) and afterbirth. Constrasted 
with the classical Central Dogma of molecular biology, in 
which DNA is recoded into transfer RNA, which ends in the 
production of proteins for the structure and functions of all 
cells. Epigenesis implies a flow of causality in the opposite 
direction. For example, numerous physiological and environ¬ 
mental factors can influence whether specific genes (DNA) 
are expressed or not. DNA is the primary molecule that 
encodes phenotypes, passing the plan for a species from one 
generation to the next. But non-DNA factors can influence 
the activation and silencing of DNA, the on/off switches. See 
Chapters 15 and 16. 

epigenetics (ep-ih-juh-NET-iks; Greek epi- = after; genesis = 
birth, origin): Changes in gene expression caused by other 
mechanisms than the direct expression of DNA via messenger 
RNA and their resulting proteins. See Chapters 15 and 16. 

episodic memory (ep-i-SOD-ic MEM-ree; from Greek 
episeidos = coming in besides): Memory for conscious expe¬ 
riences, especially those that can be explicitly recalled, such 
as times, places, events, associated emotions, and other con¬ 
textual knowledge. The formation of new episodic memories 
requires the medial temporal lobe, especially the hippo¬ 
campal region in combination with the cerebral cortex. See 
Chapters 2 and 9. 

evoked potential (EP; ee-VOKD puh-TEN-shul; from Latin 
evocare = to call forth; potentia = power): Also called event- 
related potential (ERP). A quite stereotypical electrical voltage 
pattern obtained from the brain, after averaging a time-locked 
voltage to a stimulus or other known event. Traditionally, the 
EP was obtained by averaging the stimulus-locked EEG over 
numerous trials. Though the exact brain sources of EPs are 
still debated, they are highly sensitive to cognitive and emo¬ 
tional variables. See Chapter 4, 'A Range of Useful Tools'. 

excitotoxicity (ek-SEYE-toh-tok-SIS-ih-tee; Latin, excitare = 
to arouse; toxicum = poison). Neural damaged caused by an 
excess of glutamate, the usually excitatory neurotransmitter. 
Excitotoxicity is believed to be a major cause of brain degene¬ 
ration and post-injury damage. See Chapter 16. 

executive attention (ek-ZEK-u-tiv a-TEN-shun): Also called 
voluntary, goal-directed, or top-down attention. The act of 
voluntarily focusing on one stream of conscious events while 
ignoring others. Also see selective attention, stimulus-driven 
attention. See Chapter 8. 

executive function (ek-ZEK-u-tiv FUNK-shun): Also called 
executive control or frontal lobe function. Capacities such as 
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planning, cognitive flexibility, voluntary action, abstract think¬ 
ing, rule acquisition, initiating correct actions and inhibiting 
incorrect ones, impulse control, and emotional regulation. See 
Chapter 12, 'Introduction'. 

explicit memory (eks-PLI-sit MEM-ree; from Latin expli- 
citus = clear): A type of memory involving conscious, inten¬ 
tional recollection of stored experiences, and knowledge. See 
implicit memory, implicit learning. See Chapter 2, 'Classical 
Working Memory', and Chapter 9, 'Introduction'. 

extinction (eks-TINK-shun): The process by which learned 
behaviors cease to be reinforced and therefore decline in fre¬ 
quency. In parietal neglect, after damage to the right parietal 
lobe, the ability of stimuli presented to the left side of the 
visual field to cancel those presented to the right side. See 
Chapter 13, 'The Fear System'. 

Fechner, Gustav (1801-1887): German physicist, mathema¬ 
tician, and pioneer in psychophysics. Fechner claimed to have 
solved the mind-body controversy when he demonstrated a 
general logarithmic relationship between subjective sensory 
intensity and physical stimulus intensity across many differ¬ 
ent sensory modalities. See Chapter 1, 'Some History, and 
Ongoing Debates'. 

feedback (FEED-bak): 1. In goal-guided systems, a sig¬ 
nal from the environment indicating the degree of error in 
achieving the goal. 2. In neuroscience and psychology, an 
environmental signal reflecting some neuronal event. This 
kind of neurofeedback often allows people to learn to con¬ 
trol otherwise spontaneous neuronal activities. 3. In neural 
networks, a flow of information returning an output signal to 
the input layer of the network. Some theorists make a strong 
distinction between feedback and re-entrant signaling in the 
thalamo-cortical system of the brain. See Neural Darwinism. 
Chapters 3 and 6, 'Theories of Visual Consciousness: Where 
Does It Happen?'. 

feedforward (feed-FOR-werd): 1. Signal passing from a sim¬ 
pler to a more complex stage of processing. 2. In sensorimo¬ 
tor guidance, preparing an internal action trajectory to obtain 
more precise feedback when the action is executed. This strat¬ 
egy is used in fast movement control in birds and humans, and 
even in machines like aerodynamically unstable jet planes. 
3. In neural networks, passing information from earlier to later 
layers of the network. 

fetal alcohol syndrome (FEE-tel AL-ko-hol SIN-drum): 
(FAS). Brain damage in a fetus due to the mother's alcohol 
consumption, a major health risk. See Chapter 15, 'Prenatal 
Development: From Blastocyst to Baby'. 

first-person perspective (FERST PER-son per-SPEK-tiv): 
The conscious viewpoint of the self or T. See Chapter 14, 
'Overview'. 

flanker task (FLANK-er TASK): A tool for studying visual 
attention, in which the subject is asked to respond as quickly 
as possible to a target at the center of gaze, or one located 
off-center by a known distance. The target is flanked by 


distracting stimuli, such as arrows or letters. The flanker task 
permits quantitative assessment of the subject's speed and 
accuracy in shifting attention to an expected or unexpected 
position of the target. Moving attention in an unexpected 
direction is taken to require executive attention, since it must 
override the prepared, expected shift. Flanker tasks generally 
require subjects to avoid voluntary eye movements, so that 
any change in accuracy or speed in response to a target can be 
attributed to implicit shifts of attention. See Chapter 8. 

flash suppression (FLASH suh-PRESH-en): A variant of 
the binocular rivalry task, in which an image presented to 
one eye is suppressed by a flashed image to the other eye. 
See Chapter 8. 

Fourier analysis (FOOR-ee-ay uh-NAL-a-sis): Named after 
French mathematician and physicist Joseph Fourier, who 
showed that any complex function can be decomposed into a 
finite set of sine and cosine functions. In music, for example, 
this implies that any complex sound can be decomposed into 
a set of pure tones (sine waves). Fourier analysis is routinely 
applied to decompose EEG and other complex brain signals 
into frequency bands. See Chapter 4, 'A Range of Useful 
Tools'. 

Fragile X syndrome: A genetic disorder with a range of 
physical, cognitive, emotional, and behavioral deficits. The 
most common cause of mental retardation. See Chapter 16. 

Freud, Sigmund (1856-1939): Austrian physician and neu¬ 
rologist who founded the psychoanalytic school of psychol¬ 
ogy, and a highly influential cultural figure. Freud is best 
known for his theories of the unconscious mind and for cre¬ 
ating the clinical practice of psychoanalysis. He was also an 
early neurological researcher who developed an early neural 
network model and discovered a new chemical stain - gold 
chloride - which allowed certain brainstem neurons to stand 
out clearly under the microscope. Although many of Freud's 
ideas have fallen from favor, some brain phenomena, like the 
role of the frontal lobes in regulating emotional impulses and 
goal-conflicts, are broadly consistent with his point of view. 
See Chapter 1, 'Some History, and Ongoing Debates'. 

frontal lobe (FRON-tal lobe): An large region of cortex 
located at the front of each cerebral hemisphere and posi¬ 
tioned forward of the parietal lobes and above and in front 
of the temporal lobes. The executive functions of the fron¬ 
tal lobes include the ability to anticipate the consequences 
of actions, to plan and make decisions, to speak, to over¬ 
ride inappropriate impulses and resolve conflicting goals, to 
understand the mental states of others, and to hold informa¬ 
tion in working memory. See Chapter 1, 'Some History, and 
Ongoing Debates', and Chapter 5, 'Introduction'. 

functional fixedness (FUNK-shun-el FIKS-ed-ness): A cog¬ 
nitive set that tends to block a person from novel ways of act¬ 
ing, perceiving, or solving problems. See Chapter 10, 'Explicit 
Problem-Solving'. 

functional magnetic resonance imaging (fMRI; FUNK- 
shun-el mag-NET-ic REZ-nence IH-ma-jing): A type of 
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specialized magnetic resonance imaging (MRI) that meas¬ 
ures local blood oxygenation related to neural activity in 
specific parts of the brain. fMRI helped to make cognitive 
neuroscience possible. See Chapter 4, Introduction', 'fMRI 
and PET: Indirect Signals for Neural Activity'. 

functional redundancy (FUNK-shun-el ree-DUN-den-see): 
Built-in backup functions in a system to prevent the complete 
failure of critical functions. For example, mammals have two 
lungs so that if one fails, the organism still has one lung to 
breathe. The brain has multiple redundant capabilities. See 
Chapter 3, 'Functional Redundancy'. 

fusiform face area (FFA; FYOO-ze-form; from Latin 
fusus = spindle, after its shape): A specialized region in the 
medial temporal lobe that responds selectively to visual 
faces compared to other objects. See Chapter 6, 'Functional 
Organization of the Visual System'. 

Gage, Phineas (1823-1860): A historic brain damage 
patient, whose railroad accident demonstrated remarkable 
spared cognitive capacities in spite of severe damage to the 
frontal lobes. Gage was a railroad foreman who had a two- 
foot-long thin tamping iron shot through the upper orbit 
of the left eye and out through the medial scalp, when an 
unstable dynamite charge exploded unexpectedly. Although 
Gage appeared to have no loss of perception, motor control, 
or speech, his personality changed in ways that have come to 
typify frontal lobe damage, especially a major loss of impulse 
control and long-term motivation. See Chapters 2 and 12. 

gamma-aminobutyric acid (GABA; GAM-uh uh-mee-no- 
byoo-TEER-ic ASS-id): The major inhibitory neurotransmit- 
ter in the central nervous system. It plays an important role 
in regulating neuronal and behavioral excitability, including 
sleep. GABA is needed for brain oscillations and for the regu¬ 
lation of muscle tone. See Chapter 16. 

gamma waves (GAM-a WAYVZ; third letter of Greek alpha¬ 
bet): A band of fast, low-amplitude electromagnetic wave¬ 
forms associated with wakefulness and active thinking, and 
recorded in the brain or on the scalp, apparently reflecting 
the activities of large populations of neurons. The gamma 
band is thought to be centered near 40 Hz, ranging from 25 
to 60 Hz. However, higher frequency waves are sometimes 
labeled gamma as well. Gamma is thought to reflect regional 
connectivity in the service of current tasks. See Chapter 4, 'A 
Range of Useful Tools'. 

ganglion (GAN-glee-on; Latin for swelling): A large cluster 
of neurons. The major subcortical organs may be considered 
to be ganglia, such as the basal ganglia. They are often very 
large structures, and have multiple functions. They are typi¬ 
cally composed of subdivisions, which themselves are often 
layered and folded arrays of nerve cells. See Chapter 5, 'From 
'Where' to 'What': The Functional Roles of Brain Regions'. 

gap junction (also called electrical synapse): A direct mem- 
brane-to-membrane junction between two neurons or glial 
cells. Gap junctions once were thought to be rare, but now are 
believed to exist throughout the brain, especially in GABA- 
ergic connections in the top layer of cortex. 


genome (JEE-nome; Greek, genea = generation, race): 
The full genetic blueprint specifies a biological species. See 
Chapter 16. 

genotype (JEE-no-tipe): A genome, often contrasted with 
the phenotype for a given species, which is the physiological 
expression of the genetic code. See Chapter 16. 

gestalt (gesh-TALT; German for form): 1. A perceptual 
stimulus that cannot be reduced to simple subcomponents. 
2. A branch of psychology based on the German concept of 
Gestalt , often summed up with the slogan that 'The whole is 
more than the sum of its parts'. Gestalt psychology has pro¬ 
foundly influenced the study of perception. See Chapter 6, 
'Introduction'. 

glial cell (GLEE-el SEL; from Greek glia = glue): Non¬ 
neuronal cells in the brain that support neurons, maintain 
neurochemical homeostasis, form protective myelin sheath 
around neurons, and process information. See Chapter 15, 
'Prenatal Development: From Blastocyst to Baby'. 

glutamate (Glu; GLOO-teh-mayt; from Latin gluten = 
glue): A molecule related to glutamic acid that is the most 
abundant excitatory neurotransmitter in the brain. See 
gamma-aminobutyric acid (GABA). See Chapters 3 and 16. 

gray matter: The outer layers of the cerebral cortex, as seen 
with the naked eye. Gray matter contains the cell bodies of 
tens of billions of neurons that send white-covered axons in 
many directions below the cortical mantle. See white matter. 
See Chapter 5, 'Introduction'. 

head-related transfer function (HRTF; HED reh-LAY-ted 
TRANS-fer FUNC-shun): A method for computing the loca¬ 
tion of sounds in auditory space. See Chapter 7, 'Functional 
Mapping of Auditory Processing'. 

Hebbian learning (HEB-ee-en LUR-ning): According to 
Donald O. Hebb, 'neurons that fire together, wire together'. 
That is, two neurons strengthen their synaptic links if they 
are active at the same moment. This process forms cell 
assemblies. Introduced by Donald Hebb in 1949, it is also 
called Hebb's rule. See Chapter 3, 'How Neural Arrays Adapt 
and Learn'. 

Helmoltz, Hermann von (1821-1894): German physician, 
physicist, and sensory scientist. His works on vision and 
audition are still read, but his best-known contribution was 
demonstrating the physical law of conservation of energy 
using electrical stimulation of dissected frog legs. Helmholtz 
was one of the first to propose that the visual system makes 
'unconscious inferences' that go beyond the raw light input 
to the eyes, a controversial idea in his time that has become 
widely accepted. See Chapter 1, 'Some History, and Ongoing 
Debates'. 

hemispheric lateralization (hem-is-FEER-ik lat-er-al-ih- 
ZAY-shun; from Greek hemi- (half) + sphairion = sphere; and 
Latin lateralis = side): The degree to which certain brain func¬ 
tions are performed primarily by one cerebral hemisphere, the 
most prominent being speech production on the left side for 
most people. See Chapters 5, 9, and 11. 
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hemodynamics (HEE-mo-dye-NAM-ics; from hema = 
blood; dynamos = force): The study of blood flow changes, 
particularly in the brain, as an index of local neural activity. 
See Chapters 1 and 4. 

hippocampus (hip-o-KAM-pes; Greek seahorse, from 
hippos = horse, kampos = sea monster): In the human brain, 
the hippocampi are looped structures in each of the two 
medial temporal lobes. The hippocampi are part of the lim¬ 
bic system and play basic roles in encoding and retrieving 
episodic and semantic memories and in spatial navigation. 
See Chapter 2, 'Classical Working Memory'. 

histogenesis (HIS-to-gen-eh-sis; from Greek histos = mast, 
loom, beam, web; Greek gignesthai = to be born): The forma¬ 
tion of specific cells from less differentiated progenitor cells. 
See Chapter 15, 'Prenatal Development: From Blastocyst to 
Baby'. 

homeobox (Hox) (HO-mee-oh-boks; Greek homos = same): 
DNA segments that regulate body plans in animals and 
plants. Hox genes typically regulate other genes to have their 
effects. 

homunculus (ho-MUN-cue-lus; Latin = little man): The 
distorted human body maps in the primary somatosensory 
cortex (the sensory homunculus) and in the primary motor 
cortex (the motor homunculus). The lips, hands, feet, and sex 
organs have more sensory neurons than other parts of the 
body, so the homunculus has correspondingly distorted large 
lips, hands, feet, and genitals. Each hemisphere contains a 
sensory and motor homunculus of the opposite side of the 
body. These body maps were discovered by neurosurgeon 
Wilder Penfield at the University of Montreal in the 1950s 
and 1960s. See Chapter 5, 'From 'Where' to 'What': The 
Functional Roles of Brain Regions'. 

hydranencephaly (hi-dran-en-SEF-uh-lee; Greek, hydr- = 
water; enkephalon = in the head): A fetal disorder involving 
severe damage to the brain, including failure of the cortex 
to develop normally; the missing tissue is replaced by the 
watery cerebrospinal fluid. Hydrancephaly is more severe 
than hydrocephalus, in which brain development is more 
complete, though there is a build-up of cerebrospinal fluid in 
the ventricles. See Chapter 16. 

hypercolumn (HEYE-per CAW-lum; from Greek hyper = 
above; Latin columna = top): A cluster of columns of cortex 
that often contain closely related neurons. See Chapter 5, 
'Introduction'. 

hypothalamus (hie-po-THAL-a-mus; from Latin hypo = 
below; Greek thalamos = chamber): The major neuroen¬ 
docrine organ of each side the brain, with vital roles in the 
regulation of blood nutrients, motivation, appetite control, 
and other major life functions. The hypothalamus is located 
below each thalamus just above the brainstem. It is closely 
related to the pituitary and pineal glands. See Chapter 5, 
'Growing a Brain from the Bottom Up'. 

immediate memory (ih-MEE-dee-et MEM-er-ee): Also 
called short-term memory. The ability to recall something for 


10 to 30 seconds without rehearsal. Working memory and 
sensory memories can be seen as specific kinds of immediate 
memory. See Chapter 2, 'Classical Working Memory'. 

implicit memory (im-PLI-sit MEM-er-ee; from Latin 
implicitus = obscure): Unconscious memory, which may 
arise from conscious or unconscious events. See Chapter 2, 
'Classical Working Memory', Chapter 9, 'Introduction'. 

inattentional blindness (in-uh-TEN-shun-el BLIND-ness): 
A reliable experimental phenomenon in which one is not able 
to see things that are normally clearly visible. See Chapter 8. 

inference (IN-fer-ens): Drawing a conclusion based on 
knowledge rather than direct observation. See Chapter 1, 
'Some Starting Points'. 

inferior (in-FEER-ee-er; from Latin inferus = lower): Below. 

inhibitory control (in-HIB-ih-tor-ee; from Latin in- = not; 
Latin habere = to have, to consider): Nerve impulses that act 
to damp down or stop a particular activity or response. See 
Chapter 12. 

insula (IN-soo-la; Latin for island): A structure that is hid¬ 
den in and underneath the lateral sulcus, covered up by the 
temporal and parietal lobes, and therefore appears as an 
island when the covering tissues are gently pulled away. The 
insula is involved in 'gut feelings', such as the sense of nau¬ 
sea and disgust, and possibly in emotional feelings and crav¬ 
ings. See Chapter 5, 'From 'Where' to 'What': The Functional 
Roles of Brain Regions'. 

intentionality (in-ten-shen-AL-ih-tee): The 'aboutness' of 
mental events, their ability to represent aspects of the world. 
Distinguished from intention as a mental goal. See Chapter 
14, 'Overview'. 

interaural level difference (in-ter-OR-el; from Latin inter- = 
between; auris = ear): A method of sound localization in 
which the brain detects the small difference in loudness 
between the two ears that occurs when a sound travels 
toward the head from an angle. See Chapter 7, 'Functional 
Mapping of Auditory Processing'. 

interaural-time-difference (in-ter-OR-el): A method of 
sound localization in which the brain detects the split-sec¬ 
ond delay between the time when sound from a lateral source 
reaches the near ear and when it reaches the far ear. See 
Chapter 7, 'Functional Mapping of Auditory Processing'. 

intersubjectivity (IN-ter-sub-jek-TIV-ih-tee): The sharing 
of subjective experiences, such as infants' ability to point 
to and name a toy and receive validation from an adult. A 
crucial aspect of social cognition and language learning. See 
Chapter 14, 'Overview'. 

intonation contour (in-toh-NAY-shun kon-TOOR): The 
'melody' or sing-song of normal speech. In English and other 
languages, questions typically are given a different intonation 
contour compared with affirmative statements. See Chapter 11, 
'The Sounds of Spoken Language'. 
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ionotropic receptor (eye-ono-TROP-ik ree-SEP-ter; Greek, 
ion = going; tropos = direction): A group of membrane recep¬ 
tor channels for charged particles like Ca+2, Cl-2, and Na+2. 
Ionotropic receptors may be opened or closed by neurotrans¬ 
mitters. Also called ligand-gated ion channels, or channel- 
linked receptors. 

ipsilateral (IP-si-LAT-er-al; from Latin ipse = self; latus = 
side): On the same side of the body. See contralateral. See 
Chapter 2, 'Action'. 

James, William (1842-1910): American psychologist and 
philosopher. James summarized the 19th century's stud¬ 
ies of the human mind and brain. His Briefer Psychology 
(1893) was widely used as an introductory text in psychology 
well into the 1920s. James also influenced Western stream of 
consciousness literature, educational psychology, and the 
study of religious experience and mysticism. In philosophy 
he is considered a major exponent of pragmatism, and was a 
source for European phenomenology. He was the brother of 
novelist Henry James and of diarist Alice James. See Chapter 
1, 'Some History, and Ongoing Debates'. 

lateral (LAT-er-al; from Latin lateralis = side): On the side(s) 
of the brain. See Chapter 1, 'Some Starting Points'. 

lateral geniculate nucleus (LGN; LAT-er-el jen-IK-yoo-let 
NOO-klee-us; from Latin latus = side; genu = knee-shaped; 
Latin nux = nut): A nucleus consisting of 'knee-shaped' lay¬ 
ers of cells in the thalamus. It is the primary relay center 
between the retina of the eye and the primary visual cor¬ 
tex (Area VI). See Chapter 6, 'Lunctional Organization of the 
Visual System'. 

lateral inhibition (LAT-er-el in-hi-BI-shun; from Latin 
latus = side; Latin inhibitus = restrain): The capacity of a neu¬ 
ron to reduce the activity of its neighboring cells in the same 
layer of neurons. See Chapter 3, 'Working Assumptions'. 

lateral occipital complex (LOC; LAT-er-el ox-SIP-it-al 
KOM-pleks; from Latin latus = side; Latin occiput = rearmost 
part of the skull): A region on the side of the occipital lobe that 
has a general role in visual object recognition. See Chapter 6, 
'Lunctional Organization of the Visual System'. 

lateral sulcus (LAT-er-al SUL-cus; from Latin latus = side; 
sulcus = groove): Also called Sylvian fissure or lateral fis¬ 
sure. This prominent 'valley' divides the temporal lobe from 
the frontal and parietal lobes. See Chapter 1, 'Some Starting 
Points'. 

L-dopa (el-DOH-puh; l = levorotatory, the leftward turn¬ 
ing molecule; dopa = dihydroxyphenylalanine): The first 
medical dopamine agonist used to help Parkinson's patients 
by stimulating the production of dopamine in the substantia 
nigra. L-dopa is a natural precursor molecule in the synthesis 
of dopamine. 

lexical identification (LEKS-ih-kul eye-den-tih-fih-KAY- 
shun; from Greek lexis = word): The process of assigning 
words to speech sounds. See Chapter 11, 'Introduction'. 

lexicon (LEKS-ih-con; from Greek lexis = word): The 
vocabulary of a language. See Chapter 11, 'Introduction'. 


limbic system (LIM-bik sis-tem; from Latin limbus = 
border): An ancient set of brain structures involved in emo¬ 
tion, memory, olfaction, and action control, including the 
hippocampus, amygdala, thalamus, hypothalamus, and cin¬ 
gulate gyms. The limbic system is interwoven with the endo¬ 
crine system and autonomic nervous system. See Chapter 
13, 'Introduction'. 

long-term depression (LTD; LONG TERM de-PRE-shun; 
from Latin deprimere = to press down): A lasting decrease in 
the strength of a synapse. Along with long-term potentia¬ 
tion (LTP), LTD is thought to be a synaptic basis for learning 
and long-term memory. See Chapter 3, 'How Neural Arrays 
Adapt and Learn'. 

long-term potentiation (LTP; LONG TERM puh-ten- 
shee-AY-shun; from Latin potentia = power): A long-lasting 
strengthening of a synaptic link. Along with LTD, LTP is 
thought to be the synaptic basis of learning and long-term 
memory. See Chapter 3, 'How Neural Arrays Adapt and 
Learn'. 

longitudinal fissure (lon-gi-TOOD-in-al LISH-er; from 
Latin fissus = crack, opening): The deep valley that divides 
the right and left hemispheres of the vertebrate brain. See 
Chapter 1, 'Some Starting Points'. 

magnetic resonance imaging (MRI; mag-NET-ik REZ-nence 
IH-ma-jing; Latin resonare = to sound; imago = imitation): 
Based on the spin resonance of atomic nuclei, MRI is a tech¬ 
nique used to visualize the internal structures of the body, 
including the brain. Functional MRI (fMRI) records brain 
activity and is often superimposed on the structural brain 
image obtained via MRI. See Chapter 4, 'Introduction'. 

magnetoencephalography (MEG; mag-NET-o-en-sef-eh- 
LOG-gra-fee; Greek en- + kephale = in the head; graphein = 
writing): An imaging technique based on the magnetic fields 
produced by brain activity. MEG is silent and noninvasive 
and has good temporal and spatial resolution. See Chapter 4, 
'A Range of Useful Tools', and the appendix. 

medial (MEE-dee-al): Toward the midline of the body, 
midsagittal. See Chapter 1, 'Some Starting Points'. 

medial temporal lobe (MTL; MEE-dee-el TEM-per-el 
LOBE): The bottom aspect of the temporal lobes, which are 
arranged symmetrically around the midline, and contain 
evolutionarily ancient olfactory structures, memory encod¬ 
ing and recall, and emotional functions. See Chapter 5, 'Lrom 
'Where' to 'What': The Lunctional Roles of Brain Regions'. 

memory (MEM-ree, MEM-eh-ree; from Latin memor = 
mindful): A lasting brain representation that is reflected in 
thinking, experience, or behavior. See Chapter 9. 

mental flexibility (MEN-tel fleks-ih-BIL-ih-tee; from Latin 
mens = mind; Latin flexus = bent; Latin -ibilis, from -bilis = 
capable or worthy of): Also called ability to shift cognitive 
set. The capacity to respond rapidly to unanticipated envi¬ 
ronmental contingencies. See Chapter 12, 'A Closer Look at 
Lrontal Lobe Lunction'. 
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mental rigidity (MEN-tel rih-GID-ih-tee; from Latin mens = 
mind; Latin rigidus = stiff): The inability to respond rapidly to 
unanticipated environmental contingencies, or shift cognitive 
set. Profound forms of mental rigidity produce obsessive-com¬ 
pulsive disorder. Frontal lobe damage often produces extreme 
mental rigidity. See Chapter 12, 'Frontal Lobe Dysfunction'. 

mentalize (MEN-tel-ize; from Latin mens = mind): The 
ability to understand the self and others, not just as sensory 
objects but also as subjective beings with mental states. See 
Chapter 14, 'Overview'. 

metabotropic receptor (meh-tab-oh-TROP-ik ree-SEP- 
ter; Greek, metabole = change; tropos = direction). Neuronal 
receptors that make use of the existing metabolic machinery 
of the cell to activate post-synaptic signaling. Contrasted 
with ionotropic receptors, which utilized ion channels in the 
cell membrane. 

metacognition (MET-a-cog-NI-shen; from Greek meta = 
above; Latin cognere = to know): Knowing about cognition. 
Self-knowledge about memory, perception, or voluntary con¬ 
trol. See Chapter 8. 

midsagittal (mid-SAJ-i-tal; from Latin sagitta = arrow): 
medial. The midline plane of section, going from the nose to 
the middle of the back of the head. See Chapter 1, 'Some 
Starting Points'. 

mind (from Greek menos = spirit): Those aspects of intel¬ 
lect and consciousness manifested in thought, perception, 
memory, emotion, will, and imagination, including all of the 
brain's conscious and unconscious cognitive processes. See 
Chapter 1, 'Some History, and Ongoing Debates'. 

mind-reading (from Greek menos = spirit): The attempt to 
attune one's own actions to the minds of others. Most people 
are not accurate mind-readers in most circumstances. See 
Chapter 14, 'Overview'. 

mirror neuron (MEER-er NUR-on or NYOO-ron; from 
Greek neuron = sinew, nerve): A neuron theorized to fire both 
when an animal performs an action and when it observes the 
same action performed by another. Large populations of mir¬ 
ror neurons have been found in primates and are believed 
to exist in other species, including birds. In humans, brain 
activity consistent with mirror neurons has been found in the 
premotor cortex and the inferior parietal cortex, however 
there is significant debate in the field regarding these find¬ 
ings. See Chapter 14. 

mirror neuron system (MEER-er NUR-on or NYOO-ron 
SIS-tern; from Greek neuron = sinew, nerve): The presumed 
network of mirror neurons in the human brain, which is 
believed to be needed for social cognition. See Chapter 14. 

mitochondrion (MEYE-toh-KON-dree-on; Greek mitos = 
thread, chondrion = grain): Plural, mitochondria. The energy- 
producing organelle in most animal cells. Mitochondria are 
believed to originate in blue-green algae that migrated into ani¬ 
mal cells and established a mutual dependence or symbiosis. 


Mitochondria are the chief source of the energy molecule ATP 
(adenosine triphosphate), and are indispensible for life in ani¬ 
mals. They have their own nuclei with DNA, but much of their 
DNA has migrated to the nucleus of the host cell. Because 
energy production is a fundamental life process, the metabolic 
machinery of mitochondria has also been recruited for other 
functions like neuronal signaling. See Chapter 16. 

module theory (MAH-jul THEE-eh-ree or THIR-ee; from 
Latin modulus = little measure; Greek thedria = contemplation): 
A type of theory that postulates that different brain functions 
may be localized in different regions or networks of the brain. 
See Chapter 14, 'Overview'. 

morpheme (MOR-feem; from Greek morphe = form): The 
smallest linguistic unit that can convey meaningful informa¬ 
tion by itself. In English, prefixes and suffixes are considered 
to be morphemes (e.g., 'pre-' and 'post-'). See Chapter 11, 
'Introduction'. 

motion blindness (MO-shun BLIND-nes): A symptom 
caused by injury to brain regions needed for motion percep¬ 
tion, such as area MT of the visual cortex, resulting in an ina¬ 
bility to perceive visual motion. See Chapter 6, 'Brain Areas 
Necessary for Visual Awareness: Lesion Studies'. 

multistable perception (MUL-tee-STAY-bel per-SEP-shun): 
Alternating visual perceptions of an ambiguous stimulus. See 
Chapter 6, 'Linking Brain Activity and Visual Experience'. 

myelin (MY-e-lin; from Latin my el = marrow): A sheath of 
glial cells, called the myelin sheath, surrounding the axons 
of many neurons. Myelinated axons appear white, hence the 
'white matter' of the visible brain. See Chapters 1 and 3. 

near infrared spectroscopy (NIRS; NEER in-fra-RED 
spek-TROS-kop-ee; from Latin infra = below, within; 
Greek erythros = red): A method of measuring light waves 
below the color red in the wavelength spectrum (about 
800 nm to 2500 nm). It is used to measure blood hemo¬ 
globin as an index of regional brain activity. See Chapter 15, 
'Developing Mind and Brain'. 

neglect (nuh-GLEKT): A type of brain damage to the right 
parietal lobe, in which the patient fails to consciously perceive 
or attend to the left side of objects or scenes. See Chapter 6, 
'Brain Areas Necessary for Visual Awareness: Lesion Studies'. 

neocortex (NEE-o-COR-tex; from Latin neo = new; Greek 
cort = bark): The largest and most visible part of the human 
cerebral cortex. It is the 'new' cortex from an evolution¬ 
ary point of view, as contrasted with the 'old' cortex of the 
medial temporal lobe, hippocampus, and olfactory brain. 
See Chapter 1, 'Some History, and Ongoing Debates'. 

neon color spreading (NEE-on CAW-ler SPRED-ing): A 
perceptual illusion in which white space appears to be tinted 
by proximity to colored and black lines. See Chapter 6, 'Link¬ 
ing Brain Activity and Visual Experience'. 

Neural Darwinism (NUR-el DAR-win-izm; from Greek 
neuron = nerve): A theory proposed by neuroscientist Gerald 
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Edelman that suggests that neurons develop and make con¬ 
nections following Darwinian principles. In biological evo¬ 
lution, species adapt by reproduction, mutations leading to 
diverse forms, and selection among the resulting repertoire 
of slightly different organisms. Neural Darwinism suggests 
that brains develop in similar fashion, both in the reproduc¬ 
tion, variation, and selection of developing neurons, and 
in a later stage, in the Darwinian selection of synaptic con¬ 
nections. Brains are said to be selectionist rather than instruc¬ 
tionist, unlike the program of a digital computer. See Chapter 
3, 'How Neural Arrays Adapt and Learn'. 

neural migration (NUR-el my-GRAY-shun; from Greek 
neuron = nerve): Movement of nerve cells from their place 
of origin toward their final location in the growing brain. 
See Chapter 15, 'Prenatal Development: From Blastocyst to 
Baby'. 

neural net model (NUR-el NET MO-del; from Greek 
neuron = nerve): Also known as artificial neural nets (ANNs), 
neural models are simulated, simplified models of selected 
brain functions. Most are relatively small in scale and do not 
represent the great complexity of the brain. However, they are 
important for a better understanding of how neural computa¬ 
tion might work. See Chapter 3. 

neural tube (NOOR-el TOOB): In developing vertebrate 
embryos, the cylindrical structure that will turn into the 
central nervous system. See Chapter 15, 'Prenatal Develop¬ 
ment: From Blastocyst to Baby'. 

neuroblast (NUR-o-blast; from Greek neuron = nerve; 
Greek blastos = bud, shoot): In early embryos, a dividing cell 
that will differentiate into neurons or glial cells. See Chapter 
15, 'Prenatal Development: From Blastocyst to Baby'. 

neurogenesis (NOOR-oh-JEN-uh-sis; Greek neuron = 
nerve cells; genesis = beginning): The origin and differen¬ 
tiation of neurons from progenitor cells. See Chapter 15, 
'Prenatal Development: From Blastocyst to Baby'. 

neuromodulator (NOOR-o-MOD-u-lay-ter; from Greek 
neuron = nerve; modulate is used in the sense of 'influence' 
or 'regulate'): Certain neurochemicals have very widespread 
effects in large regions of the brain. These are called neuro¬ 
modulators, whereas neurotransmitters are molecules with 
very local effects in specific synapses. See Chapters 1 and 16. 

neuron (NOOR-on or NYOO-ron; from Greek neuron = 
nerve): Nerve cells that transmit information by electrochem¬ 
ical signaling. They are the core components of the human 
brain, spinal cord, and peripheral nerves. Many different 
types of neurons exist, from sensory receptors and motor 
units and neuroendocrine cells to pyramidal neurons, which 
have long-distance axons, interneurons, which form bushy 
local connections, and a wide variety of specialized cells. See 
Chapter 3. 

neuron doctrine (NOOR-on or NYOO-ron DOK-trin; from 
Greek neuron = nerve; doctor = teacher): A theory credited 
to the Spanish histologist Santiago Ramon y Cajal, stating 


that 'the nervous system consists of numerous nerve units 
(neurons), anatomically and genetically independent'. This 
has been one of the basic assumptions of brain science for 
the past century. However, the discovery of large numbers 
of electrical synapses (gap junctions) may challenge some 
aspects of the neuron doctrine. See Chapter 1, 'Some History, 
and Ongoing Debates'. 

neuropeptide (NOOR-o-PEP-tide; from Greek neuron = 
nerve; Latin peptidia = small digestibles): A peptide is a short 
amino acid chain. Neuropeptides act as local neurotrans¬ 
mitters or neurohormones, influencing appetite regulation, 
growth hormone, and pain perception. See Chapters 1 and 3. 

neurotoxin (NOOR-oh-tok-sin; Greek, neuron = nerve cell; 
Latin, toxicum = poison): A chemical that degrades neural 
functioning, often by blocking normal neural signaling. See 
Chapter 16. 

neurotransmission (NOOT-oh-trans-MISH-en): Electrochem¬ 
ical signaling between nerve cells. See Chapters 3 and 16. 

neurotransmitter (NOOR-o-TRANS-mit-er; from Greek 
neuron = nerve; Latin trans = moving through): Chemicals 
that act to relay a signal from one neuron to the next across 
a synaptic cleft. Some neurotransmitters are packaged into 
vesicles that cluster beneath the membrane on the presynap- 
tic side of a synapse and are released into the synaptic cleft, 
where they bind to receptors located on the postsynaptic 
membrane. Release of neurotransmitters often is driven by 
action potentials in the presynaptic axon. There is a low level 
of baseline release even in the absence of an action potential. 
See Chapters 1, 3, and 16. 

neurulation (nur-uh-LAY-shun or nyur-uh-LAY-shun; from 
Greek neuron = nerve): The early development of the 
central nervous system in the vertebrate fetus. See Chapter 
15, 'Prenatal Development: From Blastocyst to Baby'. 

norepinephrine (nor-epih-NEF-rin; Greek, epi = on top of; 
nephros = kidney; -in = protein): Also called noradrenaline. 
A brain chemical with dual roles as a circulatory hormone 
and a chemical neurotransmitter. See adrenaline, noradrena¬ 
line. See Chapter 16. 

object permanence (OB-jekt PER-ma-nens): The knowl¬ 
edge that perceptual objects continue to exist even when 
they cannot be seen or touched. Object permanence begins in 
infants around 7 months. See Chapter 15, 'Developing Mind 
and Brain'. 

occipital lobe (ox-SIP-it-al lowb; from Latin occiput = a 
back bone of the skull): The occipital lobes, which contain 
the earliest visual region of the cortex, are the smallest of four 
lobes in the human cerebral cortex. See Chapters 1 and 5. 

ontogeny (on-TODG-en-ee; from Greek ontos = being; 
Greek geneia = origin): The development or course of devel¬ 
opment, especially of an individual organism. See Chapter 
12, 'Phylogeny and Ontogeny'. 
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orbitofrontal cortex (or-bit-oh-FRON-tel COR-teks; from 
Latin orbis = circle, orb, orbit, world, referring to the part of 
the brain immediately above the sockets or orbits of the two 
eyes; Greek cortex = bark): A specific frontal lobe syndrome 
where patients may show euphoria, hyperactivity, and loss of 
impulse control. See Chapter 12, 'Frontal Lobe Dysfunction'. 

orbitofrontal syndrome (or-bit-oh-FRON-tel SIN-drome; 
from Latin orbis = circle, orb, orbit, world, referring to the 
orbitofrontal cortex immediately above the sockets or orbits 
of the two eyes): A specific frontal lobe syndrome where 
patients may show euphoria, hyperactivity, and loss of 
impulse control. See Chapter 12, 'Frontal Lobe Dysfunction'. 

output functions (OWT-put FUNK-shuns): Brain processes 
controlled by the frontal lobes that include the central execu¬ 
tive, action planning, and motor output. Chapter 2, 'Classical 
Working Memory'. 

paleocortex (PAY-lee-o-COR-teks; from Greek paleo = old, 
ancient; Greek cortex = bark): An evolutionary ancient region 
of the cerebral cortex including the medial temporal lobes, 
olfactory cortex, and hippocampus. See Chapter 2, 'Classical 
Working Memory'. 

Panksepp, Jaak (b. 1943): A founder of affective neuro¬ 
science, the study of the brain basis of emotion. Panksepp 
is known for his research on laughter in other mammals (in 
response to tickling) as well as high-frequency separation 
distress cries, such as occurs when rat mothers and their 
pups are separated. See Chapter 13, 'Panksepp's Emotional 
Brain Systems'. 

parahippocampal place area (PAIR-a-HIP-o-KAMP-el; 
from Greek para = before; see hippocampus (PPA)): A region 
near the hippocampus that responds more strongly to land¬ 
scapes and visual scenes than to isolated objects like houses 
or faces. See Chapter 6, 'Functional Organization of the Visual 
System'. 

paralinguistic (PAIR-uh-lin-GWIS-tik; from Greek from 
para = near; Latin lingua = language): The nonverbal aspects 
of linguistic communication, such as voice intonation, gesture, 
social distance, and eye contact. See Chapter 11, 'Meaningful 
Statements'. 

pathology (path-OL-uh-gee; from Greek pathologia = study 
of the emotions): Something abnormal. The study of the essen¬ 
tial nature of diseases and especially of the structural and func¬ 
tional changes produced by them. See Chapter 12, 'Frontal 
Lobe Dysfunction'. 

periaqueductal gray matter (PAG; peh-ree-ah-kwa- 
DUK-tel GRAY MA-ter; from Greek peri = near; Latin 
aquaeductus = water channel): The gray matter that sur¬ 
rounds the midbrain and brainstem tube (aqueduct) that car¬ 
ries cerebrospinal fluid between the ventricles and the spinal 
cord. See Chapter 13, 'The Fear System'. 

parietal lobe (puh-REYE-uh-tl lowb; from Latin parietalis = 
relating to walls): A large cortical region located above the 


occipital lobe and behind the frontal lobe. The parietal 
lobe integrates sensory information from different modalities, 
and contains constantly updated maps of the position of the 
body and nearby objects. See Chapters 1 and 5. 

parietal neglect (puh-REYE-uh-tl neh-GLEKT; from Latin 
parietalis = relating to walls): A condition in which dam¬ 
age to the right parietal lobe causes the left side of egocen¬ 
tric space to become unconscious. See Chapter 13, 'The Fear 
System'. 

Pavlov, Ivan Petrovich (1849-1936): Russian physiologist. 
He was awarded the Nobel Prize in Physiology or Medicine 
in 1904 for research pertaining to the digestive system. He is 
best known for describing 'classical conditioning', in which 
an arbitrary stimulus like the sound of a bell came to signal 
the coming of a biological stimulus like food, thereby elicit¬ 
ing salivation. Pavlov's proposal that conditional reflexes are 
the basic unit of all human learning is no longer generally 
believed. However, Pavlovian conditioning is widely used in 
research and is relevant to clinical issues such as food pref¬ 
erences and aversions, and perhaps anxiety disorders. See 
Chapter 1, 'Some History, and Ongoing Debates'. 

Penfield, Wilder (1891-1976): A neurosurgeon and resear¬ 
cher in Montreal who performed pioneering work in epilep¬ 
tic surgery. Before operating, he performed exploratory brain 
stimulation in awake patients (who were free of pain using 
only local anesthetic in the surgical opening). Thus patients 
could report their experiences upon electrical brain stimula¬ 
tion. Penfield and coworkers were able to determine functions 
of the human brain that were previously only approach¬ 
able via postmortem studies of brain damaged patients. See 
Chapter 4, 'A Range of Useful Tools'. 

perceptual filling in (per-SEP-choo-el FIL-ing in): A gen¬ 
eral feature of sensory perception in which the brain fills in 
missing parts of a visual object or scene, often far beyond the 
direct sensory input. See Chapter 6, 'Linking Brain Activity 
and Visual Experience'. 

perceptual memory (per-SEP-choo-el MEM-ree, MEM-eh- 
ree; from Latin memor = mindful): Long-lasting changes in 
one's ability to perceive the world, e.g., the ability to perceive 
the sounds of speech, and to recognize visual objects under 
changes in orientation and lighting. See Chapter 2, 'Classical 
Working Memory'. 

peripheral nervous system (PNS; per-IF-er-el NUHR-vus 
SIS-tem): The extensive network of neurons outside of the 
brain and spinal. The PNS includes sensorimotor neurons 
below the neck, and autonomic neurons that innervate the 
smooth musculature of the digestive tract, heart, and circula¬ 
tory system. See Chapter 5, 'Introduction'. 

perseveration (per-sev-er-AY-shun; from Latin perseverare = 
persist): A symptom involving the inappropriate and uncon¬ 
trollable repetition of a specific word, phrase, or gesture. See 
Chapter 10, 'Explicit Problem-Solving'. 
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phenotype (FEE-no-tipe; Greek phainein = to show; typos = 
type): Any physiological trait of an organism that expresses the 
DNA blueprint, or genotype. See Chapter 16. 

phoneme (FO-neem; from Greek phone = sound): In 
human languages, the smallest lexically distinctive category 
of sound, such as consonants and vowels. See Chapter 7, 
'Speech Perception'. 

phonemic deficit (fo-NEEM-ic; from Greek phone = sound): 
A form of aphasia in which speech sounds cannot be identi¬ 
fied in terms of phonemic categories. See Chapter 7, 'Speech 
Perception'. 

phosphene (FOS-feen; from Greek phos = light; Greek 
phainein = to show): Light spots in the visual field that are 
induced by direct mechanical, electrical, or other stimula¬ 
tion of the retina or visual cortex. Phosphenes have also been 
reported by meditators, during sensory isolation, and under 
the influence of drugs, such as hallucinogens. See Chapter 6, 
'Manipulation of Visual Awareness'. 

phytogeny (feye-LODG-en-ee; from Greek phylon = race; 
Greek geneia = origin): The evolutionary history of a kind 
of organism or a genetically related group of organisms, as 
distinguished from the development of the individual organ¬ 
ism. See Chapter 12, 'Phylogeny and Ontogeny'. 

pituitary gland (pi-TOO-uh-tehree gland; from Latin 
pituita = phlegm, from the former belief that the pituitary 
secreted phlegm): The 'master gland' of the body, also called 
the hypophysis. An endocrine gland about the size of a pea 
that appears to hang from the hypothalamus at the base of 
the brain. The pituitary works with the hypothalamus to reg¬ 
ulate developmental stages and homeostasis. See Chapter 5, 
'Growing a Brain from the Bottom Up'. 

planum temporale (PLAH-num tem-por-AHL-eh; from 
Latin planum = a flat surface; Latin temporalis = of the tem¬ 
ple): A part of the auditory cortex involved in sound analysis 
and particularly speech perception. See Chapter 7, 'Functional 
Mapping of Auditory Processing'. 

plasticity (plas-TI-SI-tee; from Greek plastikos, from pla- 
ssein = to mold, form): The ability of the brain to adapt and 
reorganize to new environmental inputs or demands, or fol¬ 
lowing brain damage. See Chapter 7, 'Music Perception'. 

pons (PONZ; Latin, pons = bridge): A prominent anterior 
bulge in the brainstem. The pons relays sensory information 
between the cerebellum and the forebrain and spinal cord, 
helps to control sleep and wakefulness, and regulates respi¬ 
ration among other functions. It also generates REM sleep 
signals that are interpreted by the cortex as visually vivid, 
narrative dreams. See Chapter 5, 'Growing a Brain from the 
Bottom Up'. 

positron emission tomography (PET; POH-zi-tron ee- 
MISH-en tom-OG-reh-fee; Latin emitterre = to send out; 
Greek tomos = section; Greek graphein = writing): Positrons 
are positively charged subnuclear particles, typically pro¬ 
duced by a particle accelerator. PET is a low-level radioactive 


imaging technique that allows the computational extraction 
of brain or body slice maps, from which a three-dimensional 
image can be constructed. See Chapter 4. 

postcentral gyrus (post-SEN-tral JEYE-res; from Latin 
post = behind; Latin gyrus = ridge): A protruding fold in the 
parietal lobe of the human brain immediately behind the 
central sulcus. It includes the primary somatosensory cortex, 
the first cortical map of the body senses, also called the 
sensory homunculus, which represents the opposite or 
contralateral side of the body. 

posterior (pos-TEER-ee-er; from Latin post = after): Behind. 
In brain anatomy, posterior is synonymous with caudal. 

prefrontal cortex (pree-FRON-tal KOR-teks; from Latin 
prae = in front of; frons = the forehead; Greek cort = bark): 
The large, forward portion of the frontal lobes, not includ¬ 
ing the motor cortex. Prefrontal cortex includes executive 
functions and Broca's area, and is sometimes called 'the 
organ of civilization'. See Chapter 2, 'Classical Working 
Memory'. 

primary motor cortex (PRIE-mar-ee MO-ter KOR-teks; 
from Latin primus = first; Greek cort = bark): The brain 
region that directly controls skeletal (voluntary) muscles. It 
corresponds to the motor homunculus, and works in close 
association with other sensory body and motor maps, such 
as the premotor cortex. 

primary somatosensory cortex (PRIE-mar-ee so-MAT-o- 
SENS-ery KOR-tex; from Latin primus = first, most impor¬ 
tant; soma = body; sensus = sense; Greek cort = bark): The 
sensory homunculus (body map), located on the postcentral 
gyrus of the cortex, it is the first cortical area for the body 
senses. 

primary visual cortex (PRIE-mar-ee VIZH-oo-el KOR-teks; 
from Latin primus = first, most important; Latin visus = sight; 
Greek cort = bark) (also called VI): The first cortical map of 
the visual system, located in the occipital lobe. See Chapter 
6, 'Functional Organization of the Visual System'. 

problem space (PROB-lem SPAYS): A graph of the decision 
points in problem solving, often in the form of a tree struc¬ 
ture. See Chapter 10, 'Explicit Problem Solving'. 

procedural memory (pruh-SEE-der-el MEM-ree, MEM- 
eh-ree; from Latin procedere = a way of doing things; Latin 
memor = mindful): A form of implicit memory equivalent 
to skill memory, or knowing how to do a task. It appears to 
be largely unconscious. This type of memory is often very 
durable. See Chapter 2, 'Classical Working Memory'. 

process specificity (PRAH-ses spes-i-FIS-ih-tee): The claim 
that a cognitive process may utilize a variety of different 
brain regions. See Chapter 12, 'A Closer Look at Frontal Lobe 
Function'. 

proprioception (PRO-pree-o-SEP-shun; from Latin proprius = 
one's own; Latin perceptio = perceiving): Senses relating to the 
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self, including one's location in space, based both on internal 
and external sensory input. 

proteome (PRO-tee-OME; by analogy to genome): The entire 
set of proteins expressed by a genome. 

pure word deafness: The inability to comprehend spoken 
word meanings, while still being able to hear sounds, to speak, 
read, and write. Associated with bilateral damage to the poste¬ 
rior superior (rear upper) temporal lobes or their subcortical 
connections. See Chapter 7, 'Speech Perception'. 

radial unit model (RAY-dee-el YOO-nit M AH-del): A model 
of neural migration proposed by neuroscientist Pasko Rakic 
that asserts that in the developing cerebral cortex, the cells 
are created at the base of each cortical column and each new 
cell migrates past its predecessors. See Chapter 15, 'Prenatal 
Development: From Blastocyst to Baby'. 

Rakic, Pasko (b. 1933) (rah-KEECH): A neuroanato¬ 
mist who showed that neural migration occurs radially as 
well as rostrally, like the outflowing spokes of a forward- 
moving wheel. See Chapter 15, 'Prenatal Development: From 
Blastocyst to Baby'. 

Ramon y Cajal, Santiago (1852-1934): Spanish pioneer in 
microscopic studies of the brain. He often is considered to be 
the founder of brain science. Many of his detailed drawings 
of brain tissue slices are still presented today. See Chapter 1, 
'Some History, and Ongoing Debates'. 

receptive field (ree-SEP-tiv FEELD; from Latin recipere = to 
take): The receptive field of a nerve cell in the visual system, 
for example, is the region of the visual field that can activate 
or inhibit the firing of the cell. The receptive field of a retinal 
receptor is therefore different from the receptive field of a 
higher level cell tuned to detect motion or visual object iden¬ 
tity. Analogous receptive fields have been found for visual 
attention in the parietal lobe. Receptive fields are found in 
other sensory systems as well, such as the auditory and soma¬ 
tosensory systems. See Chapter 3, and Chapter 6 for more dis¬ 
cussion on receptive fields in the visual system. 

re-entrant connectivity (ree-EN-trent con-ec-TIV-e-tee): Most 
brain connections are bidirectional, in that activity at point 
A triggers activity at point B and vice versa. See Chapter 3, 
'Working Assumptions'. 

re-entry (ree-entry): In Neural Darwinism, the resonant 
looping between two neurons or arrays of neurons, so that 
neuron A activates neuron B and vice versa. Re-entry can also 
take place between neuronal populations. It is believed to be 
the primary signaling mechanism among brain regions, and 
therefore closely related to brain rhythms. 

reflex circuit (REE-fleks SIR-kut): Also called a reflex arc, 
this is the relatively simple pathway that mediates a reflex 
action. The most common example is the knee-jerk (or patel¬ 
lar tendon) spinal reflex, which occurs even when the spinal 
cord is isolated from the brain. However, spinal reflexes can 
be quite fast, complex, and coordinated, and may interact 
with the brainstem and the vestibular (balance) system, as in 


the case of a cat reorienting its body during a fall. Normally 
reflexes work in close coordination with voluntary control 
via the frontal lobes, cerebellum, and basal ganglia. Cranial 
reflexes like the pupillary reflex are under the joint control 
of autonomic, visual, and emotional regions of the brain. See 
Chapter 3, 'Working Assumptions'. 

region of interest (ROI; REE-gen of IN-trest): A region of 
the brain selected to be tested in a brain imaging study, in 
order to make statistically valid predictions about expected 
activity in that region. See Chapter 4. 

replication (rep-lih-KAY-shun): 1. Copying of a genome 
in inheritance. 2. Redoing an experiment by using the exact 
same methods in order to verify the findings with a new sub¬ 
ject group. 

reticular formation (reh-TIC-u-ler for-MAY-shun; from 
Latin reticulum = network): A part of the brainstem that is 
involved in the sleep-waking cycle and many other functions. 
It receives collateral input from all sensory and motor systems, 
as well as from higher-level brain structures. It is evolutionar- 
ily one of the oldest parts of the brain. See Chapter 5, 'From 
'Where' to 'What': The Functional Roles of Brain Regions'. 

reticulofrontal disconnection syndrome (reh-tic-yoo-lo- 
FRON-tel dis-kon-EK-shun SIN-drome; from Latin 
reticulum = network; from = the forehead): A deficit of 
executive functioning thought to be caused by damage to the 
pathways connecting the frontal lobes to the 'network-like' 
(reticular) structures of the brainstem and mesencephalon. 
See Chapter 12, 'Frontal Lobe Dysfunction'. 

retina (REH-tin-a): The array of light receptors lining the 
inner surface of the eye. Light striking retinal receptors (rods 
or cones) trigger a chemical reaction that evokes a change in 
electrical potential across the cell membrane. This may trig¬ 
ger activity in retinal ganglion cells that project their axons 
to make up the optic nerve, which terminates in the visual 
relay nucleus of the thalamus. See Chapter 6, 'Functional 
Organization of the Visual System'. 

retrograde amnesia (RET-ro-grayd am-NEE-zhuh; from 
from Latin retrogradus = going back; Greek a-mnesia = 
without memory): A form of memory loss extending before 
the time of brain injury. Contrasted with anterograde amne¬ 
sia. See Chapter 9, 'Amnesia'. 

retrovirus (RET-roh-VEYE-rus; Latin, retro = backward; 
virus = venom). A virus that acts in reverse from the con¬ 
ventional flow of DNA-RNA-protein. Retroviruses are RNA 
parasites that can be integrated into the DNA of host cells. 
The HIV virus was the first retrovirus to be studied in great 
detail. Retroviruses are utilized to insert or delete specific 
genes into cellular DNA. 

reversal conditioning (ree-VER-sel kon-DISH-un-ing): A 
technique using classical conditioning in which an animal is 
first conditioned to fear one stimulus and not to fear another 
one. In the next phase, these conditions are switched, and the 
animal must adapt to a rapid reversal of the cues for fearful 
events. See Chapter 13, 'The Fear System'. 
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sagittal (SAJ-i-tal; from Latin sagitta = arrow): Any section 
of the brain that runs parallel to the medial or midline cut. See 
Chapter 1, 'Some Starting Points'. 

scotoma (skeh-TO-ma): A damaged or missing part of the 
retina, or of a higher-level neuronal map of the visual field. 
Scotomas often are filled in with information from regions 
adjacent to the missing neurons. See Chapter 6, 'Manipula¬ 
tion of Visual Awareness'. 

second-person perspective (SEH-kond PER-son per-SPEK- 
tiv): A person-to-person point of view, considered as a meet¬ 
ing between two subjective perspectives, self and other. See 
Chapter 14, 'Overview'. 

selectionism (suh-LEK-shun-izm): A brain theory based on 
Neural Darwinism. In biological evolution species adapt 
by reproduction, mutations leading to diverse forms, and 
selection among the resulting repertoire of slightly different 
organisms. Neural Darwinism suggests a similar selectionist 
process in the growth of neurons and their synaptic connec¬ 
tions. See Chapter 3, 'How Neural Arrays Adapt and Learn'. 

selective attention (suh-LEC-tiv a-TEN-shun): The abil¬ 
ity to pay attention to one aspect of the environment while 
ignoring competing stimuli. This may occur voluntarily, as in 
choosing to read an interesting book while sitting on a noisy 
bus, or when one sensory experience is biologically or per¬ 
sonally significant. See Chapters 2 and 8. 

semantic deficit (seh-MAN-tic DEF-ih-sit): A form of apha¬ 
sia involving a loss of a particular domain of meaning, such 
as animal names. See Chapter 7, 'Speech Perception'. 

semantic memory (seh-MAN-tic MEM-ree or MEM-er- 
ee): A type of declarative memory that involves meanings, 
factual beliefs, categories, and other general knowledge 
going beyond specific experiences. See Chapter 2, 'Classical 
Working Memory'. 

semantics (seh-MAN-tiks): The study of meaning in lan¬ 
guage. See Chapter 11, 'Introduction'. 

sensory system (SEN-suh-ree SIS-tern): Part of the ner¬ 
vous system responsible for processing sensory informa¬ 
tion. A sensory system consists of sensory receptors, neural 
pathways, and mostly posterior cortex involved in sensory 
perception. The classical senses have many subsenses, like 
pain and even tickle sensations, light receptors in the eye that 
trigger melatonin as a sleep hormone, the balance sense, and 
the like. Not all sensory systems yield conscious experiences; 
blood pressure, for example, which is sensed by hypotha¬ 
lamic neurons, is rarely conscious. The classical senses begin 
with receptor surfaces containing many millions of receptors, 
such as the retina and the basilar membrane. See Chapter 2, 
'Introduction'. 

sequential grouping (seh-KWEN-shul GROOP-ing): One 
way in which the human auditory system organizes sound 
into perceptually meaningful elements. If sound properties 
are repeated in the same sequence, they may be grouped 


together. See Chapter 7, 'Functional Mapping of Auditory 
Processing'. 

serotonin (seh-roh-TOE-nin; Latin, serum = blood, tonus = 
tension): A neurotransmitter that was originally discovered 
as a blood-pressure-regulating hormone. Serotonin has mul¬ 
tiple functions in the brain and spinal cord, and serotonin 
dysfunction is believed to be responsible for some brain 
disorders. 

shadowing (SHA-doe-ing): An experimental technique 
to study selective attention in which subjects repeat speech 
immediately after hearing it. With practice, subjects can 
learn to shadow speech with a lag time of less than a second. 
The shadowing task is sufficiently demanding that other 
streams of speech cannot be understood at the same time. See 
Chapter 8. 

simulation theory (sim-yoo-LAY-shun THEE-eh-ree): The 
notion that we sometimes mentally simulate or imitate what 
we believe others to be experiencing, in order to understand 
them. See Chapter 14, 'Overview'. 

simultaneous grouping (SEYE-mul-TAY-nee-us GROOP- 
ing): If two sounds have common onsets (beginnings) and 
offsets (endings), they may be grouped together. One way 
in which the human auditory system organizes sound into 
meaningful elements. See Chapter 7, 'Functional Mapping of 
Auditory Processing'. 

sound localization (SOUND lo-cal-ih-ZAY-shun): Identi¬ 
fying the location of a sound, often based on binaural dis¬ 
parities of timing and loudness between the two ears. See 
Chapter 7, 'Functional Mapping of Auditory Processing'. 

source memory (SORS MEM-ree or MEM-er-ee): Memory 
for the specific time, place, and circumstances when an event 
was experienced. See Chapter 9, 'Varieties of Memory'. 

spectrograph (SPEK-tro-graf; from Latin spectrum = 
appearance; Greek graphon = to write): A machine developed 
during World War II by Bell Telephone Laboratories to ana¬ 
lyze sound signals and produce a picture called a spectro¬ 
gram, showing sound frequencies over time. See Chapter 7, 
'Speech Perception'. 

spiking code (SPI-king CODE): The rate and pattern of 
action potentials, which may transmit useful information in 
the brain. See Chapter 3, 'Arrays and Maps'. 

stimulus-driven attention (STIM-u-lus DRI-vn a-TEN- 
shun): The capture of attention by salient stimuli. See 
Chapter 8. 

Stroop test (STROOP test): Named after American psy¬ 
chologist John Ridley Stroop, who first wrote about this 
phenomenon in English in 1935. When the name of a color, 
such as blue, green, or red, is printed in a color differing from 
that expressed by the word's meaning (e.g., the word red is 
printed in blue ink), a subject has more difficulty naming the 
color of the word and is slower and more prone to errors than 
when the meaning of the word is congruent with its color. 
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This phenomenon is known as the Stroop effect. The Stroop 
effect is useful in activating conflict-related regions of the 
brain, and generalizes well to related tasks, like the 'emo¬ 
tional Stroop'. See Chapter 8. 

superior (soo-PEER-ee-er): Above. In the human brain, it is 
synonymous with dorsal. 

supratemporal plane (SOO-pra-tem-per-el PLANE): A flat 
region of cortex in the Sylvian fissure, where primary and 
secondary auditory cortex and parts of Wernicke's area 
are located. See Chapter 5, 'From 'Where' to 'What': the 
Functional Roles of Brain Regions'. 

Sylvian fissure (SIL-vee-en FISH-er): Also called the lateral 
sulcus or lateral fissure. This prominent 'valley' of the cor¬ 
tex divides the frontal lobe and parietal lobe above from the 
temporal lobe below. See Chapter 5. 

synapse (SIN-aps): Synapses are tiny gaps between neu¬ 
rons that communicate by way of chemical neurotransmit¬ 
ters. Synapses are a basic computational element of the brain, 
a kind of traffic control point for the flow of information. The 
brain has tens of billions of neurons, but it has many trillions 
of synapses. 

synaptic cleft (sin-AP-tic CLEFT): The space between two 
neurons that can communicate with each other via neuro¬ 
transmitters. See Chapter 3. 

synaptic pruning (sin-AP-tik PROO-ning): The selec¬ 
tive loss of synapses in the brain when some potential 
connections are not utilized. See Hebbian learning, neural 
Darwinism. See Chapter 15, 'The Developing Brain: A 
Lifetime of Change'. 

synaptogenesis (sin-AP-toe-GEN-eh-sis): The birth of syn¬ 
apses in the brain. See Chapter 15, 'The Developing Brain: A 
Lifetime of Change'. 

syntactic analysis (sin-TAK-tik uh-NAL-ih-sus): The iden¬ 
tification of grammatical structures from words, phonemes, 
and morphemes. See Chapter 11, 'Introduction'. 

syntax (sin-TAKS): The rules and regularities of sentences 
in natural languages. See Chapter 11, 'Introduction', and 
'Syntax, Nesting, and Sequencing'. 

Talairach coordinates (tal-AY-rahk co-ORE-din-etz): A 
precise three-dimensional coordinate system for the human 
brain that can localize any point in the brain with millimeter 
precision. See Chapter 5, 'Introduction'. 

temporal envelope (TEM-por-al EN-ve-lope; from Latin 
tempor = time): The rising and falling intensity of speech sig¬ 
nals over time. See Chapter 7, 'Speech Perception'. 

temporal lobe (TEM-por-al lobe): The temporal lobes are 
parts of the cerebral cortex that are involved in visual percep¬ 
tion, hearing and speech perception, and memory encoding 
and recall. They emerge from the sides of the cortex, beneath 
the lateral sulcus. In profile, if the human brain resembles 
a boxing glove, the temporal lobes would be the thumb of 
each side. The temporal lobe envelops the hippocampus and 


amygdala and is therefore involved in emotion and memory 
formation as well. The medial temporal lobe (most easily seen 
from the bottom perspective of the brain) is ancient paleocor¬ 
tex, including olfactory cortex. See Chapters 1 and 5. 

teratogen (ter-AT-e-jen): A chemical or other factor that 
causes developmental malformations. See Chapter 15, 
'Prenatal Development: From Blastocyst to Baby'. 

terminal (TER-mi-nul): The distal end of an axon. See 
Chapter 3, 'Introduction'. 

thalamo-cortical system (THAL-a-mo COR-ti-kel SIS- 
tem; from Greek thalamos = chamber; Greek cort = bark): A 
central hub in the brain involving the cortex and thalamus, 
allowing signal traffic to flow flexibly back and forth in both 
directions. See Chapter 5, 'Introduction'. 

thalamus (THAL-a-mus; from Greek thalamos = room, 
chamber): A pair of symmetric egg-shaped structures in 
the brain that provide the main cortical input hub and cor- 
tico-cortical traffic hub. See Chapter 1, 'Some History, and 
Ongoing Debates'. 

theory of mind (THEE-eh-ree or THIR-ee of MIND): 
The ability to attribute mental states - beliefs, desires, 
intentions - to others. See Chapter 14, 'Overview'. 

theory (THEE-eh-ree or THIR-ee): The notion that children 
develop implicit theories of other people over time, much as 
scientists produce theories over many years of testing and 
development. See Chapter 14, 'Overview'. 

theta waves (THAY-ta WAVZ; eighth letter of Greek alpha¬ 
bet): Theta waves are regular electromagnetic waveforms 
with a typical frequency of 3.5 to 7.5 Hz. One role of theta is to 
coordinate hippocampal memory encoding with the neocor- 
tical sites of synaptic memory storage. Theta is also involved 
with the reverse process of episodic retrieval. Theta rhythms 
are thought to involve many neurons firing in synchrony, 
driven by cholinergic neuromodulation. 

third-person perspective (THIRD PER-son per-SPEK-tiv): 
A public viewpoint on evidence. Public evidence for some 
hypothesis is typically required in science. See Chapter 14, 
'Overview'. 

transcranial magnetic stimulation (TMS; trans-CRAY-nee- 
el mag-NET-ic stim-yoo-LAY-shun): A relatively noninvasive 
method using powerful electromagnets outside of the head 
to stimulate or inhibit cortical neurons. TMS shows good 
temporal and spatial resolution. See Chapter 4, 'A Range of 
Useful Tools'. 

transcription (tran-SKRIP-shun; Latin trans = across, 
scrihere = to write): The copying of DNA into messenger 
RNA. See Chapter 16. 

translation (trans-LAY-shun): The conversion of messenger 
RNA into active proteins. 

triune brain (TRY-oon BRAYN; Latin, tri = three): A 
broad model proposed by Paul MacLean to characterize the 
human brain in terms of three major evolutionary stages of 



634 


GLOSSARY 


development, including the reptilian complex, limbic sys¬ 
tem (mammalian), and neocortex (primates and other large¬ 
brained mammals, with a large frontal lobe expansion in 
humans). See Chapter 13. 

unconscious perception (un-CON-shus per-SEP-shun): 
Sensory stimulus processing without awareness of the stimu¬ 
lus. See Chapter 6, 'Manipulation of Visual Awareness'. 

utilization behavior (yoo-til-ih-ZAY-shun be-HAYV-yor): 
Also called field-dependent behavior. A frontal lobe disorder 
in which victims may imitate the actions of an examining 
physician or find it irresistible to use tools placed in front of 
them. See Chapter 12, 'Frontal Lobe Dysfunction'. 

ventral (VEN-trel; from Latin venter = the belly): The lower 
part of a brain structure, inferior. 

ventricles (VEN-trik-lz): Four small cavities in the brain 
containing circulating cerebrospinal fluid. The ventricular 
walls have been found to be sites for neural stem cells. See 
Chapter 5, 'Growing a Brain from the Bottom Up'. 

ventromedial prefrontal cortex (ven-tro-MEE-dee-el pree- 
FRON-tal KOR-teks; Latin venter = the belly; medialis = in 
the middle): The bottom midline structures of the frontal 
lobe, especially in humans and other primates. This region, 
extending backward from the top of the nose, is involved in 
emotions, infant-mother bonding, fear, and risk in decision 
making. See Chapter 12. 

verbal dyspraxia (VER-bl dis-PRAK-see-uh; Greek, dys- 
= impaired; praxis , action): A condition in which speaking is 
impaired. See Chapter 16. 

verbal rehearsal (VER-bel ree-HER-sel): Mental repetition 
of words to be remembered, using the 'inner speech' compo¬ 
nent of working memory. Inner speech involves a spontaneous 
commentary on current concerns, goals, and emotions. See 
Chapter 2, 'Classical Working Memory'. 

Vesalius, Andreas (1514-1564): Belgian physician who 
produced the first accurate atlas of human anatomy in 1543, 
called On the Fabric of the Human Body. Vesalius' book was 
a major milestone in the Renaissance rediscovery of science 
and medicine. See Chapter 1, 'Some History, and Ongoing 
Debates'. 

vesicle (VES-i-cl; from Latin vesicula = small bladder): The 
small bubbles filled with of neurotransmitter molecules that 
travel through the axon to the synaptic terminals, where 
they fuse with the synaptic membrane to release neuro¬ 
molecules into the cleft when an action potential occurs. 
Neurotransmitters then diffuse across the synapse to trigger 
depolarization of the postsynaptic membrane, ultimately 
leading to another axonal spike. Vesicles are essential for the 
propagation of signals between neurons and are constantly 
recreated by the cell. See Chapters 3 and 16. 

visual agnosia (Vl-zhoo-el ag-NO-zhe; from Greek 
agnosia = lacking knowledge): A condition in which a per¬ 
son has difficulty recognizing objects because of damage to 


object-recognition regions of the cortex, such as the inferior 
temporal lobe. See Chapter 6, 'Brain Areas Necessary for 
Visual Awareness: Lesion Studies'. 

visual backward masking (Vl-zhoo-el BAK-werd MAS- 
king): A conscious visual image can be 'erased' by a sub¬ 
sequent visual event, such as a cross-hatch display, even 
though the conscious event is not physically blocked from 
reaching the retina. See Chapter 8. 

visual phantoms (VIZH-oo-el FAN-tems; from Anglo- 
French fantasme = phantasm): A form of perceptual filling in 
in which visual forms seem to float in front of the surround¬ 
ing shapes and colors that create the illusion. See Chapter 6, 
'Linking Brain Activity and Visual Experience'. 

visuospatial sketchpad (vizh-oo-oh-SPAY-shul SKECH- 
pad): The ability to hold visual and spatial information 
momentarily in working memory. See Chapter 2, 'Classical 
Working Memory'. 

vocoder (VO-CO-der; from voice + coder): A technology 
developed by Bell Telephone Laboratories to transmit highly 
filtered speech with minimum loss of intelligibility. The tech¬ 
nology forms the basis for cochlear implants. See Chapter 7, 
'Speech Perception' and Box 7.1. 

volition (vuh-LI-shun): Voluntary control of actions, as 
contrasted with automatic control, as in the case of highly 
practiced habits. Many brain disorders involve a loss of vol¬ 
untary control. See Chapter 12. 

Wearing, Clive (b. 1938): A prominent British classical 
musician who suffered a viral brain infection in his for¬ 
ties that destroyed both hippocampi and some frontal lobe 
regions. Wearing's case has become well known due to the 
efforts of his wife, Deborah Wearing, to raise public aware¬ 
ness of such medical conditions. Wearing lives in a single, 
blindered moment, without the ability to store information 
for later recall. Despite his memory problems, he is still able 
to play the piano and conduct musical pieces he knew well 
before the brain injury. See anterograde amnesia. See Chapter 
2, 'Classical Working Memory'. 

Wernicke, Carl (1848-1905): German physician and dis¬ 
coverer of a selective cortical region for speech comprehen¬ 
sion. This region is now referred to as Wernicke's area, and 
the associated deficit is known as Wernicke's or receptive 
aphasia. Patients with this deficit cannot understand speech, 
including their own, but produce fluent-sounding (but not 
usually meaningful) speech. See Chapter 1, 'Some History, 
and Ongoing Debates'. 

Wernicke's aphasia (WER-nik-ees AY-PHAY-zha; from 
a = without; Latin phasia = utterance): See Wernicke, Carl. 
See Chapter 1, 'Some History, and Ongoing Debates'; Chapter 7, 
'Speech Perception'. 

Wernicke's area (WER-nik-ees AIR-ee-a): An area of the 
upper posterior temporal lobe that is needed for language 
comprehension. See Chapter 1, 'Some History, and Ongoing 
Debates'. 
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white matter: In the brain white matter consists of dense 
bundles of myelinated axons, which connect various gray 
matter areas of the brain to each other. White matter is named 
for the appearance or massive numbers of myelinated nerve 
axons, which appear to form the visible core of brain struc¬ 
ture. See Chapter 1, 'Some History, and Ongoing Debates'. 


working memory (WUR-king MEM-ree or MEM-er-ee): A 
cognitive capacity for storing and manipulating novel infor¬ 
mation over 10 to 30 seconds. Working memory includes 
central executive, working storage, verbal rehearsal, and the 
visuospatial sketchpad. See Chapter 2, 'Classical Working 
Memory'. 
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Abducens nucleus, 135f 
Abstract concepts, 359-360 
Acetylcholine, 510, 511, 514, 540t 
Acetylcholinesterase, 514 
Achromotopsia, 178f 
Acoustical analysis, 372, 372f 
Action, 54-57 
Action lobes, 399-400 
Action planning, 36 
Action potentials, 65 
Action recognition, 453 
Activation, PFC, 410 
Active maintenance, 409 
Active memory, 347 
Actor-centered decision-making, 411 
Adaptation, 29, 88 
Adaptation, neural, 169 
Adaptive decision-making, 411 
Adaptive organ, 524 
Adenosine, 510 
ADHD see Attention deficit 
hyperactivity disorder 
Adolescence, 492-503 
Adrenalin, 512 
Affective blindsight, 430-431 
Affective empathy, 460-462 
Affective qualities of stimuli, 430 
Affect-relevant information, 426 
Age of maturity, 406-407 
Aggression, 271 
Agnosia 

apperceptive, 178 
associative, 179 
auditory, 215-216 
moral, 406 
object, 406 

visual, 178-181, 365-366 
Agonist, 511 
Akinotopsia, 178f 
Alcohol, 5t, 6, 474 
Alert response, 498f 
Alerting network, 408 


Allocentric mapping, 75 
Alpha blocking, 270 
Alpha waves/rhythms, 89-91, 105- 
106, 106f, 107t, 109f, 269-270 
Alpha/theta feedback, 297-298 
Alpha/theta training, 298f 
Altered states, 295-301 
Alzheimer's disease, 315, 329, 353 
neuroimaging studies of, 562 
Amacrine cells, 160-161 
Ambiguity 

decision making, 411-412 
language, 373 
visual, 80,182 

American Academy of Child and 
Adolescent Psychiatry, 298 
Amine group, 519 
Amino acids, 519 
Amnesia, 37, 311-317 
anterograde, 313-314 
Clive Wearing, 36-40, 311-317 
HM (patient), 37, 37f, 38f, 55-57, 
300, 311-313 

implicit learning in, 316-317 
implicit memory in, 314-315 
organic, 311, 312f, 314 
procedural memory in, 315 
psychogenic, 313-314 
retrograde, 314 

Amodal symbol systems, 360f 
AMPA, 534 
AMPA receptor, 545f 
Amplification and selectionist 
systems, 24 

Amygdala, 136,149f, 151, 307, 404 
afferent pathways, 427,427f 
efferent pathways, 425-426, 427, 
427f 

nuclei of, 428f 
terminology, 13 
Anagrams, 389 
Anencephaly, 521 


Anger, 586f 

Animal magnetism, 15-16 
Animal studies, 100-101, 103f 

delayed match to sample, 332f, 348 
Anisotropic environment, 558f, 579 
"Anatomy Lesson of Dr. Tulp 
(Rembrandt), 4f, 13 
ANT (Attention Network Task), 273, 
408,490f 
Antagonists, 511 

Anterior cingulate cortex (ACC), 52, 
273-274,296,353-354,406, 
410-411,458 

cognitive/emotion regions, 412f 
Anterior commissure, 131 
Anterior PFC (APFC), 400 
Anterograde amnesia, 313-314 
Anticipation of reward see Reward 
Aphasia, 223-224 
Brocas, 223-226 
types of, 19f 

Wernickes, 19, 223-224, 225 
AplYsia, 510-513 
Apparent motion, 186-187 
Apperceptive agnosia, 178 
Arcuate fasciculus, 20, 225, 375 
Area MT, 166-167,167f, 188-189 
Area V4,166 

Arousal threshold and sleep, 284 
Arrays, 71-81 
choice-points, 79-81 
defined, 69, 71 

sensory and motor systems, 75-76 
temporal codes, 76-79 
top-down/ expectation-driven 
processing, 81 
two-way connections, 75 
Artificial neural networks (ANNs), 29, 
68, 91 

Ascending (afferent) pathways, 203 
Ascending reticular activating system 
(ARAS), 152f 
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Associative agnosia, 179 
Associative process, 143 
Associative recall, 259 
Attention, 270-278 
ANT task, 273,408,490f 
benefits/costs of, 273 
vs. conscious experiences, 278 
consciousness and, 270-278, 301f 
deficit, 181-182 
ADHD, 415, 416-417 
divided, 330-331 
emotional influences, 432-433 
executive, 53-54, 407-408 
flanker task, 271-274 
Posners model, 53f 
Stroop Task, 273 
fear studies, 429-430 
focal, 26-27, 346-347 
joint, 457f, 491 
and learning, 258-259 
model of, 274-278 
neuroimaging studies of, 407-408 
outcome of, 271 

overlap with working memory, 
348-349 

paying attention, 258 
Posner flanker task, 271-274 
and reward, 440 
selective, 34, 276, 277 
limited capacity functions, 41,42f 
shared, 449-450, 456-457, 457f 
stimulus-driven, 276 
voluntary, 407-408 
vs. automatic, 276,407-408 
brain regions involved, 276-277 
Attention deficit hyperactivity 

disorder (ADHD), 415, 416-417 
Attentional blink, 287-288 
Attentional capture, 276 
Attentional control mechanisms, 276 
Attentional loop, 417f 
Attentional zoom capacity, 347 
Audition, 33-34 
Auditory agnosia, 215-216 
Auditory attention, 212-213, 214f 
Auditory awareness, 232-235 
during sleep and sedation, 233 
Auditory cortex, 204-209, 205f 
anatomy, 204-206 
columnar organization, 208 
functional mapping, 209 
neurophysiology, 206-209 
tonotopic organization, 201, 208 
Auditory imagery, 233-235 
Auditory language, 227f, 380f 
Auditory nerve, 7t, 200-202 


Auditory object recognition, 213-216 
grouping, 216 

noisy environments, 216-217 
source, 216 

stream segregation, 216 
what and where processing streams, 
210-218 

Auditory pathways, 71 f, 72f, 

203-204 

Auditory processing, 200-202 
functional mapping, 209-218 
Auditory scene analysis, 196 
Auditory sentence processing, 
379-380,394f 

Auditory stream segregation, 216 
Autistic spectrum disorders (ASD), 

450 

Automatic attention, 276, 407-408 
Automatic behaviors, 243 
Automatic processes, 70 
Automaticity, 52-53, 259-261, 355 
Autonoetic consciousness, 327 
Autonomic nervous system, 36, 
149-150 

Auto-receptors, 526 
Autosomal dominance, 516 
Awareness, 26-27 
Axonal fibers, 43-44 
Axonal spikes, 526, 527 
Axons, 64, 66f 

Back-propagation, 84 
Balint's syndrome, 182 
Band of frequencies, 261, 270 
Baron-Cohen, Simon, 447, 449-450, 
450f 

Basal ganglia, 136,138f, 151, 404 
speech control, 384, 386f 
Basilar membrane, 201 
Basolateral complex, 426-427 
Behavioral inhibitor, 531 
Behavioral testing, 486f 
Behaviorism, 23f 
Behaviors, 406 
Benson, Frank, 402 
Berger, Hans, 269 
Beta endorphin, 540t 
Beta waves, 105-106,107t, 261, 270 
Between-subjects designs, 575-578 
Binocular disparity, 164-165,165f 
Binocular rivalry, 103f, 121-122, 
182-185,265-266 
Biochemical individuality, 514 
Biochemical pathways, 403 
Biological motions, 171-172 
Biologically plausible neural nets, 68 


Bipolar cells, 160-161,161f 
Birth defects, 521 
Bistable figures, 183f 
Bistable patterns, 182 
Bistable perception, 182-183 
Blastocyst, 468, 469f 
Blind spot, 160-161,185,186f 
Blindsight, 176-177 
affective, 430^31 
Block design, 575, 576f, 578f 
Blood-brain barrier, 512, 513f 
Blood-oxygen level dependent activity 
(BOLD), 559-563, 576-577 
conscious perception, 576-577 
fear, 428 

finger activity, 558f 
functional neuroimaging, 113-114, 
115,115f 
physics of, 117f 
practiced skills, 260-261 
signals, 117f, 573 
truth-telling, 123,123f 
typical response, 116f 
word counting, 122-123 
Body maps, 143-144 
Body space, 146 
BOLD 

see Blood-oxygen level dependent 
activity 

Bottom-up attention, 271 
Bottom-up processing, 76 
Bottom-up relationships, 427 
Brain 

as adaptive organ, 524 
aging, 574f 

blood-brain barrier, 512, 513f 
development see Development 
directions, 145f 
evolution, 516-518 
evolution and personal history, 
133-134 

geography, 129-133 
growth, 133-136 
imaging see Neuroimaging 
lateral view, 10, Ilf 
major landmarks, 10-13,130f 
medial view, 10, Ilf 
nervous system, 128-129 
primates vs. mice, 518f 
regions and functions, 136-153 
cerebral hemispheres, 136-141 
interconnectivity, 149-150, 245 
major lobes, 145-149 
output/input, 143-145, 250 
satellites of subcortex, 151-153 
sagittal section, 10,12f 
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Brain activity 

conscious, 121-122 
contemporary study of, 27-28 
unconscious, 121-122 
Brain injury/dysfunction 
and causal inferences, 124 
dorsal parietal areas, 181-182 
double dissociation, 53f, 124 
early, and developmental plasticity, 
503-505 

frontal lobes, 413-417 
recovery of function, 176 
speech perceptual areas, 223-226 
traumatic, 416 
ventral object areas, 178-181 
visual areas, 176-177 
visual deficits 

cortical color blindness, 177-178 
damage to dorsal parietal areas, 
181-182 

damage to early visual areas, 176 
damage to ventral object areas, 
178-181 

extrastriate lesions, 177-178 
motion blindness, 177 
VI and blindsight, 176-177 
visual agnosia, 178-181 
Brain navigation software, 99f 
Brain plasticity, 568f 
Brain rhythms, 76-79, 89-91, 107t 
of circadian states, 244-247, 244f, 
245f, 247f 
in deep sleep, 286 
fast, and waking, 261-262 
frequencies, 105-106,106f, 248f 
maps and, 250 

Brainstem, 135,135f, 136,138f, 404 
Brainwave entrainment, 296 
Breath suspensions, 299-300 
Broadbent, Donald, 41, 42f 
Broadcasting, global, 70f, 152, 288, 
289-290,294 
Broca, Pierre-Paul, 18-20 
Broca's aphasia, 223-224, 225-226 
Broca's area, 35 
biologic aspects and language, 

375, 377 

brain geography, 131 
brain regions/functions, 20f, 21 f, 145, 
224f, 386-387,395f, 452^53 
gamma synchrony, 266-267 
genetics and, 516 
identification of, 18-20, 223-224 
inner speech and, 49 
language and biologic aspects, 

375 


lesions to, 124f, 377 
magnetoencephalography, 114f 
Brodmann, Korbinian, 130-131 
Brodmann's areas, 130-131, 205f, 380f 
defining PFC, 403 
language-related, 380f 
Buber, Martin, 460 

Cajal, Santiago Ramon y, 13,16-17, 

16f, 21 

Calcium, 510, 531f, 533f, 549 
Callosotomy, 140-141 
Cannabinoid, 510 
Capacity limits, 41 
dual task limits, 41-43 
focus of attention, 347 
measuring working memory, 44-46 
reasons for narrow, 44 
very large brain capacities, 43-44 
very large-capacity functions, 43t 
Care, 424,425t 
Catecholamines, 532f 
Caton, Richard, 563 
Caudate nucleus, 136,138f 
Causal evidence, 188 
Causal inferences, and brain injuries, 124 
Causation, 122-124 
Cell adhesion, 544, 547f 
Cell assemblies, 35, 69 
Cells, 519-520, 526 

Center-surround receptive fields, 162f, 
164f 

Central auditory system, 202-209 
auditory cortex, 204-209 
auditory pathways, 71 f, 72f, 203-204 
Central Dogma of Molecular Biology, 
The, 519-520, 549 
Central executive, 34-35, 36, 50-54 
Central nervous system (CNS), 
128-129,128f 

Central sulcus, 11,130f, 143f, 144-145, 
146f, 376f, 403 

Centromedial nucleus, 150f, 426-427 
Cerebellum, Ilf, 129f, 135,151, 404 
speech control, 384, 386f 
Cerebral cortex, 10,140f, 143 
Cerebral hemispheres, 129,136-141, 
140f 

Cerebrospinal fluid (CSF), 136, 512 
Changeable faces, 456 
Channel blockers, 533 
Charcot, Jean-Paul, 296 
Chess, 351 
Childhood, 492-503 
Choice-points, 79-81, 350, 373 
Cholesystokinin, 540t 


Chromatin/chromatin reshaping, 
522-523 

Chronic inflammation, 538, 550f, 551 
Chronic sleep deprivation, 243 
Chunking, 43, 351 
Churchland, Patricia, 26 
Cigarette smoking, 474 
Cingulate cortex, 404 
Cingulate gyrus, 148-149,457f, 458f 
Cingulate sulcus, 149f 
Circuits, 69-71 

Classically conditioned responses, 423 
Cochlea, 201 
Cochlear nucleus, 207f 
Cocktail party effect, 216-217 
Cognates, 30 
Cognition, 119-121 
actor-centered, 411-412 
vs. affective empathy, 460-462 
biology and, 14-16 
executive functions, 405-406 
frontal lobes, 400,402-403 
social, 406-407 

Cognition-emotion interactions, 431 
Cognitive appraisal, 423 
Cognitive control, 496-499 
Cognitive development, 479-503 
Cognitive impairment, 474-476 
Cognitive neuroscience, 8-10,127 
Cognitive operations, 405 
Cognitive set, 405 
Cognits, 418 
Collateral sulcus, 149f 
Color blindness, 177-178 
Color perception, 159,166f 
Coma patients, 246 
Common fate, grouping by, 159 
Complexity and consciousness, 292-293 
Computed tomography, 96 
Conceptual deficits, 361-362 
in patient EW, 364-365 
Conceptual priming, 310, 314-315 
Conceptual working memory, 45 
Conditioned on probability, 481 
Conditioned stimuli (CS), 437 
Conduction aphasia, 20f, 225, 375 
Cones, 160-161 
Confabulation, 319, 327, 339 
Conflict, 412 

Connectional selectional repertoire, 24 
Connectionism, 83 
Connectivity patterns, 292f 
Conscious brain events, 121-122 
contemporary study of, 27-28 
overlap with working memory, 
348-349 
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Conscious perception 
BOLD in, 576-577 
conscious vs. unconscious vision, 
28f 

single-unit recording, 103f 
Consciousness, 287-301 
19th century research, 20-25 
alpha/theta feedback, 297-298 
attention and, 278, 30If 
autonoetic, 327 
biological theory of, 23-25 
brain activity, 302t 
brainwave entrainment, 296 
conversion disorder, 296 
evidence for unusual states, 295-301 
and experiencing self, 293-294 
feelings of knowing, 363-367 
global workspace function, 287-291 
hypnosis, 296 

mediated waking cognition, 

242-255 

circadian states, 244-247 
maps and rhythms, 250 
need for waking/sleep/ 
dreaming, 243-244 
neuron synchronization, 252-254 
synchrony for gain control, 
254-255 
tasks, 257t 

thalamocortical activity, 247-248 
thalamocortical core, 248-250 
two-way connections, 252 
meditation and similar practices, 
299-300 

neurofeedback, 296-297 
noetic, 327 

out-of-body experiences, 301 
prefrontal cortex in, 331-338 
primary vs. higher-order, 24-25, 292 
recent research (Koch), 241-242 
reentrant signaling and complexity, 
249f, 25If, 252, 291-293 
reportable events, 294 
rhythmic entrainment of 
brainwaves, 298-299 
selection process, 241, 246 
sensorimotor rhythm, 297-298 
trained oscillations, 297-298 
Conservation, 428, 511, 512 
Consolidation, 57-59, 322-323 
vs. multiple trace theory, 325 
rapid consolidation, 323-324 
systems consolidation, 323, 324-325 
time course of, 325f 
Consolidation hypothesis, 57-58, 58f 
Consonants, 381 


Constructive perception, 185-187 
Contingency relationships, 82 
Contralateral ear, 207 
Contralateral side, 54-57 
Contralateral visual field, 163 
Convergent measures, 10 
Converging evidence, 407 
Conversion disorder, 296 
Copernicus, Nicolaus, 12f, 13 
Co-registration, 115-118 
Coronal section, 12f 
Corpus callosum, 12f, 129,149, 377 
described, 11,139-141 
diffusion tractography, 94f, 122f 
Correlation, 89, 90f, 122-124,182,187 
Cortex, 127-133 
cell assemblies, 25If 
front-back division, 143-145 
functions of, 34f 

interconnectivity with thalamus, 
149-150, 245, 247-250 
lobes of, 129-131,145-149 
terminology, 13 
see also Prefrontal cortex (PFC) 
Cortical activity, 354-356 
Cortical circuits, 530f 
Cortical color blindness, 177-178 
Cortical columns, 132 
Cortical control, 141 f 
Cortical feedback, 188-189 
Cortical layers, 131f, 132,132f 
Cortical lobes, 129-131,145-149 
Cortical nucleus, 426-427 
Cortical thickness 
and age, 495f 

and intelligence, 478-479,481f 
Corticocortical connections, 247 
Cortico-cortical system, 249-250 
Corticospinal tracts, 250 
Corticotrophin releasing factor (CRF), 
540t 

Counting Stroop task, 122,122f 
Covert speech, 50f 
Cowan, Nelson, 346f 

focal attention and immediate 
memory, 346-347 
Cranial nerves, 149-150 
Creative key, 119-121 
Creoles, 374-375, 378 
Crick, Francis, 241 
Cross-correlogram, 78, 79f 
Cross-modal spatial information, 35 
Current concerns, 367 

Da Vinci, Leonardo, 13 
Damasio, Antonio, 402,406 


Dance training, 298f 
Dark adaptation, 160 
Darwin, Charles, 15,15f 
Darwinian principles, 85-87, 516, 517 
Data interpretation, 113 
DB (patient with blindsight), 176 
Decibel (dB) system, 199 
Decision-making 
adaptive, 411 

neuroimaging studies of, 411-412 
veridical, 411 

Declarative knowledge, 38 
Declarative learning, 258, 261 
Declarative memory, 330-331, 542 
Decussation, 71 f 
Deep sleep 

arousal threshold, 284 
brain rhythms, 286 
EEG, 244 

long-range gamma phase-locking, 
289f 

memory replay and consolidation, 
284-286 

mental activities in, 283-284 
Slow Wave Sleep (SWS), 247-248 
up and down states, 245, 282-286 
Degeneracy and selectional systems, 
24 

Degradation, 539, 551 
Deixis, 294 
Deja vecu, 327 
Deja vu, 327 

Delayed match to sample (DMTS), 
45-46, 59-60, lOlf, 348 
in macaque, 202f, 332f 
Dell, Gary, 49 

Delta waves/rhythms, 89-91,105-106, 
107t 

Dementia, 416 
Dendrites, 64 
Dendritic spines, 67f 
Dendritic trees, 477 
Depression, 416 
Depth electrodes, 100,102f, 104 
Descartes, Rene, 13,14f, 139, 241 
Descending (efferent) pathways, 
203-204 

Deterministic situations, 411 
Development, 465 
additive changes, 477 
birth defects, 521 
brain, 467 

cognitive processes, 479-503 
evolution, 514-518 
executive functions, 485-488 
functional framework, 483f 
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of human brain, 479f 
infants and learning, 481-482 
information acquisition, 487-488 
language, 481-482, 484, 493f 
mapping brain changes, 480f 
postnatal, 477-478 
prenatal, 467^477 
anatomy, 468-470 
epigenesis, 468 

nature and nurture, 15,474-476 
neural migration, 470-473 
prenatal hearing, 476-477 
regional differences, 478-479 
rise and fall sequence, 478 
study techniques, 466-467 
Developmental cognitive 

neuroscience, 465-466 
Developmental plasticity, 503-505 
DF (patient with apperceptive 
agnosia), 178-179 
Diamagnetic blood, 573 
Diencephalon, 470f 
Differentiation, 89, 246 
Diffusion, 539, 551 
Diffusion tensor imaging (DTI), 121, 
579-582 

Diffusion tractography, 94f, 121f, 122f, 
375,579-582 
Digit-span task, 335-336 
Direct electrical brain stimulation, 96, 

107- 113 

transcranial magnetic stimulation, 

108- 113 

Disconnection aphasia, 20, 20f 
Discourse, 372f 
Disinhibition, 415 
Distance and seven orders of 
magnitude, 4-6 
Distributed neural nets, 87 
Diversity and selectionist systems, 24 
DNA, 511 f, 519f 

information processing in cells 
(Central Dogma), 519-520 
time scales and evolution, 515-516 
Domain specificity, 221 
Dopamine, 6,403, 511, 537f, 540t 
molecule, 5f 
neurons, 437f 

reward pathways, 436,436f 
Dopaminergic neurons, 540 
Dopaminergic synapse, 570f 
Dorsal cochlear nucleus, 135f 
Dorsal lateral nucleus, 150f 
Dorsal parietal areas, damage to, 
181-182 

Dorsal pathway, 167-169 


Dorsal vagal motor nucleus, 135f 
Dorsolateral prefrontal cortex 

(DL-PFC), 52, 318f, 331-338, 
400, 496 

problem-solving, 353-354 
Dorsolateral prefrontal syndrome, 414 
field-dependent behavior, 414-415 
mental rigidity and perseveration, 
415 

personality changes, 414 
Dorsomedial prefrontal cortex 
(DM-PFC), 459 

Dorsomedial thalamic nuclei, 403, 404 
Dostoievsky, Fydor, 295 
Double dissociation, 53f, 124,179 
Down-regulation, 434-435 
Dreaming 
brain activity, 302t 
defined, 279-280 
EEG patterns, 244-247 
lucid, 282 

need for timely, 243-244 
REM dreams, 279-282 

dreaming as conscious state, 
281-282 

features of, 2811 
memory consolidation, 282 
Drug abuse, 295-301,439-440,475f 
effects on behavior, 440t 
genetic effects, 523 
prenatal, 474 
wanting, 440f 
Dual task limits, 41^3 
Dyadic interactions, 449 
Dynamic causal modeling, 123 
Dynamic core, 24-25 
Dysarthria, 124 

Eagleman, David, 141f 
synesthesia, 141-142 
Ear, 194f, 200f 
Early life experience, 406 
Early warning system, 422 
Echolalia, 414-415 
Echopraxia, 414-415 
Ectoderm, 468 

Edelman, Gerald M., 23f, 85-87, 516 
biological theory of consciousness, 
23-25 

Edinger-Westphal nucleus, 135f 
EEG (Electroencephalography), 8-10, 
101-106, 559-563 
alpha rhythms, 269-270 
deep electrodes, 562-563 
electrode net, 490f 
of epileptic seizure, 245f, 246 


event-related (evoked) potentials, 
44,106,109f, 484, 559 
musical meaning, 109f 
N170,489-491,499-500 
fast rhythms and waking, 261-262 
intracranial, 104, 245, 562-563 
meditation, 562f 

millivolt/microvolt readings, 104 
raw data, 560f, 561f 
regular rhythms, 89-91 
rhythms of circadian states, 

244-247, 244f, 245f, 247f 
sampling populations of neurons, 
102-106 

scalp electrodes, 102 
spontaneous activity, 559 
Effective connectivity, 582-583 
Efficient neural coding, 161-162 
Egocentric mapping, 75 
see also Mapping 
Einstein, Albert, 350-351 
Elaborative learning, 30 
Electrical self-stimulation (ESS), 
435-436 

Electrically evoked memories, 319-321 
Electricity and nerves, 16f 
Electrocorticograph (ECoG), 102f, 104, 
105f 

Electroencephalogram, 559 
Electroencephalography 
see EEG 
Emotion, 421 
biology and, 14-16 
feelings of, 425 
inhibition, 434-435 
Emotional and social behavior, 433 
Emotional brain systems, 423-425 
Emotional face responses, 502f 
Emotional learning, 431 
Emotional memories, 282 
Emotional significance of stimuli, 
429-430 

Emotional Stroop test, 432^33 
Empathy, 446 

Encoding, 65f, 87f, 309-310 
Endocannabinoid, 510 
Endocannabinoid activity, 543f 
Endoderm, 468 
Endogenous events, 46, 69 
Entorhinal cortex, 307 
Environment and genes, 517f, 522-523 
Enzymes, 511-512 
Epigenesis, 468 
Epigenetics, 523f 
learning, 524, 548-549 
regulation, 522 
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Epilepsy, 295 
automatisms, 243 
EEG of seizure, 245f, 246 
gamma synchrony, 266-267 
treatments for, 297-298 
Epinephrine, 512 

Episodic learning, 258, 261, 542-543 
Episodic memory, 325-329, 330t 
in amnesia, 316-317 
contemporary study of, 26-27 
loss of, 37 

as mental time travel, 317-318 
overlap with working memory, 
348-349 
retrieval, 308f 
Episodic recall, 26-27 
Error detection, 355f 
Event-related (evoked) potentials 

(ERPs), 44,106,109f, 484,559 
musical meaning, 109f 
N170,489-491,499-500 
Event-related design, 575, 576f, 578f 
Evidence 
causal, 188 
correlational, 187 
Evolution, 133,401-402, 514-518 
EW (patient), 364-365 
Excitatory neurons/connections, 65-68 
Excitatory neurotransmitters, 528, 551 
Excitotoxicity, 528, 551 
Executive activity, 357f 
problem-solving, 350-354 
Executive attention, 53-54,407^08 
flanker task, 271-274 
Posner's model, 53f 
Stroop Task, 273 
Executive control, 418 
Executive effort, 52-53 
Executive functions, 405-406 
cognitive operations, 405 
development of, 485-488 
motor control, 405f, 410-411 
self-awareness, 402-403 
Executive network, 408 
Exercise, physical, 550f 
Existing knowledge, 481 
Exogenous input, 69 
Expectation-driven processing, 81 
Experiencing self, 293-294 
Expertise hypothesis, 180 
Explanatory inferences, 7 
Explicit cognition, 26-27 
Explicit learning, 310 
consciousness and, 261 
medial temporal lobe in, 329-331 
prefrontal cortex in, 334-335 


Explicit memory, 310-311 
emotional modulation, 431^132 
medial temporal lobe in, 329-331 
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Freud, Sigmund, 27 
Friederici, Angela D., 379f 
spoken sentence processing, 
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connections, 404 
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expression and regulation, 518-524 
birth defects, 521 

development-controlling, 521-522 
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epigenetics, 548-549 
gene programming, 522 
information processing in cells 
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sensory and motor systems, 75-76 
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and language acquisition, 335 
prefrontal cortex in, 334-335 
Implicit memory, 310-311, 314-315 
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and emotional learning, 431 
prefrontal cortex in, 334-335 
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Learned processes, 70 
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Ligand-gated ion channels, 536f 
Liking, 438, 439f 
Limbic cortex, 131,148 
Limbic system, 148,422 
Limited capacity features, 346 
Limited capacity tasks, 42-43t 
Linguistic memory, 35 
Listening 

evolution of, 384-386 
selective, 216 


Load, memory, 334 
Local Field Potential (LFP), 92 
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Longitudinal fissure, 10-11 
Longitudinal relaxation (atom 
realignment), 572 

Long-range gamma phase-locking, 
289f 
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Long-term potentiation (LTP), 83, 84f, 
321-322, 322f, 543, 544f, 545f, 
546f 

Lucid dreaming, 282 
Luria, Alexander, 402,405,418 
Lust, 424,425t 

Macaque 

animal studies, 100-101, 451f 
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Mental workload, 354-356 
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Necker Cube, 44, 80, 80f, 182,183f 
Negative feedback loops, 525 
Neglect 

parietal, 428-429 
visual, 181,192 

Neocortex, 41,131,134,149, 306,423 
and memory, 307 
Neon color spreading, 186 
Nerve impulse, 16-17 
Nervous system, 128-129 
Nesting, 389f, 390-391 
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selectionism in brain, 291 
Neural migration, 470-473 
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direct electrical stimulation, 107-113 
direct measurements, 96 
EEG, 8-10, 101-106, 559-563 
alpha rhythms, 269-270 
deep electrodes, 562-563 
electrode net, 490f 
of epileptic seizure, 245f, 246 
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spontaneous activity, 120f 
study designs, 575 
subtraction method, 117-118, 
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116f 

regions of interest, 115-118 
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time-space tradeoff, 96-98 
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functions, 540t, 54If 
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Neuronal signaling, 510-511, 524-527 
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chemical self-regulation, 525-526 
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on-off switches, 524 
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gamma synchrony and visual 
features, 264-267 
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Olfactory stimuli, 426 
Olivocochlear (OC) neurons, 202f 
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constructive, 185-187 
emotional influences, 432-433 
neuroimaging studies of, 407-408 
and stimulus, 190 
Perception-action cycle, 409 
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Perceptual grouping, 159-160 
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Perceptual priming, 314-315 
Perceptual symbol systems, 360 
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8-10, 113-121, 568-570 
classical findings, 116f 
image analysis, 569-570 
subtraction method, 118f, 125f 
visual vs. auditory brain activity, 
116f 

Phantom limb sensation, 568f 
Phase coherence, 261 
Phase locking, 252, 261 
Phi (consciousness measure), 292 
Phonemes, 218, 221, 381-383, 388f, 484 
Phonemic deficit, 224-225 
Phonology, 376 
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Photopigment, 160 
Phylogenesis, 133, 401-402 
Physical exercise, 550f 
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Pidgin, 378 
Pitch, 199-200 
Pituitary gland, 135-136 
Place principle, 201 
Planum temporale (PT), 206, 217 
in sound decoding, 209-210 
Plasticity, 231-232, 548 
due to deprivation, 231 
due to expertise, 232 
due to learning, 231-232 
neural, 340, 547 
Play, 425t, 426 
Play aggression, 271 
Point-light animation, 171f, 172f 
Pons, 135,135f 

Pons-Geniculate-Occipital (PGO) 
theory, 280 

Populations of neurons, 100,102-106 
Positron emission tomography 
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Posner flanker task, 271-274 
Posner's model of executive attention, 
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Posterior parietal cortex (PPC), 
260-261, 293-294, 301, 317 
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Postsynaptic cell receptors, 532 
Post-traumatic stress disorder (PTSD), 
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Practice effect, 260-261, 357,410 
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Vladimirovich, 559 
Predetermined epigenesis, 468 
Predictable routines, 259-261 
Predictions, 81,437 
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clinical conditions, 416-417 
connections, 404 
definition of, 403 
evolution, 517-518 
frontal lobe functions, 402-403 
and memory, 331-338 
regions of, 400,400f 
thalamic connections, 404f 
Prefrontal Cortex , The (Fuster), 409 
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Premotor cortex, 404 
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anatomy, 468-470 
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nature and nurture, 15, 474-476 
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Presynaptic neurons, 532 
Primary consciousness, 24-25, 292 
Primary visual cortex, 163,164-166 
Priming, 310, 314-315 
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Proactive vs. reactive, 402 
Probabilistic epigenesis, 468 
Problem space, 350, 352f 
Problem-solving, 345 
chunking, 43, 351 
executive control in, 350-354 
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focal attention and immediate 
memory, 346-347 
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semantic memory, 357-359 
sudden insight, 367f 
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356-362 

working memory, 347-349 
Procedural memory, 310-311 
in amnesia, 315 
Production of neurochemicals, 
531-532 

Projection areas, 47 
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Propositions, 387, 393f 
Prosencephalon, 470f 
Prosody, 379-380, 391 
Prosopagnosia, 179-180 
Protein synthesis, 57-58 
Proteome, 519 
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Pruning, 478, 516 
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PubMed, 30 
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Pyramidal neurons, 75f, 132-133 
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Rage, 424,425t 
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Ranganath, Char an, 317f 
episodic memory, 317-318 
Reaction time, 7t, 45, 448-449 
Reading, connectivity patterns, 568 
Real time learning, 482 
Realistic thinking, 256-257 
Reappraisal, 434 
Recall, 258-259 
Receipt of reward see Reward 
Reception, 65f 

Receptive fields, 73f, 160-161 
center-surround, 162f, 164f 
lateral interactions and, 70-71 
mapping, 207, 208 
on-center off-surround, 161 
Receptors, 69-71,534f 
detect signaling, 534-535 
ionotropic, 529 
metabotropic, 529 
traffic control points, 533 
Recognition, 258-259 
Reconsolidation, 434 
Reconstructing memory, 308 
Recovery of function, 176 
Recycling, 539 
Re-entry, 24-25 
connectivity, 69 
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291-293 

Reflective consciousness, 423 
Reflex circuits, 69-71 
Regions of interest, 115-118 
Regressive events, 478 
Regulation 

see Expression and regulation 
Regulatory DNA, 520, 522, 531 
Rehearsing and memory, 346 
Reinterpretation, 434 
Relaxation (atom realignment), 572 
Relaxation response, 299-300 
Release, synaptic, 532 
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memory colsolidation, 282 
memory consolidation, 282 
Rembrandt, 4f, 13 

Remembering vs. knowing, 325-327 
Reorienting response, 499f 
Repetitive events, 259-261 
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Representation, 88, 446 
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Resting brain, 118-119 
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syndrome, 415-416 
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fading, 174-175 
Retinotopic map, 164 
Retrograde amnesia, 314 
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Reward, 435^140 
anticipation, 436 
and attention, 440 
and drug use, 439-440 
effects of agents/drugs, 440t 
errors in prediction, 437 
vs. learning, 438-439 
liking, 438,439f 
receipt of, 436 
reinterpretation of, 436-437 
wanting, 438^39,439f 
Rhinal cortex, 307 
Rhinal sulcus, 149f 
Rhombencephalon, 470f 
Rhythm, brain see Brain rhythms 
Rhythm, music, 229 
Rhythm, speech, 391 
Rhythmic entrainment of brainwaves, 
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Rods, 160-161 
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Routine, 259-261,405 
Rugg, Michael D., lllf 

neuroimaging correlates of 
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how babies learn, 481-482 
Sagittal section of brain, 10,12f 
Salience maps, 274-275 
Salivatory nuclei, 135f 
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Schizophrenia, 416 
Schore, Allan, 406 
Schwann cells, 66f 
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Scotoma, 188 
Scoville, William, 55-56 
Second messengers, 534f, 535 
Second person perspective, 446-447, 
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Second repertoire of neurogenesis, 516 
Second-order false belief, 461 
Seeking, 424, 425t, 435 
Selection, 85-87, 409 
Selection and consciousness, 241, 246 
Selectionism in brain, 291 
Selectionist theory of consciousness, 
23-24 

Selective attention, 34, 276, 277 
limited capacity functions, 41,42f 
Selective listening, 216 
Self-awareness and executive 
function, 402-403 
Self-control, 415 
Self-focus groups, 434-435 
Semantic deficit, 224-225 
Semantic memory, 35, 309f, 325-329, 
330t 
loss of, 37 

problem-solving, 357-359 
Semantic networks, 360-361 
Semantic priming, 310 
Semantic processing in infants, 494f 
Semantics, 372, 376, 377f, 390-391 
Sensorimotor rhythm (SMR), 297-298 
Sensory buffer, 34 
Sensory hypothesis, 489 
Sensory systems, 75-76 
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Sentences, 376 

auditory processing, 379-380, 394f 
dependency relations, 392 
Sequencing and syntax, 389, 390-391 
Sequential grouping of sound, 216 
Serotonin, 510, 540t 
Seven orders of magnitude, 4-6 
Shared attention, 449-450, 456-457, 
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Sherrington, Sir Charles, 136-139 
Short-term memory, 324f 
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Silent lobes, 400 
Similarity, grouping by, 159 
Simulation theories, 449 
Simultanagnosia, 182 
Simultaneous grouping of sound, 216 
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Single photon emission computer 

tomography (SPECT), 557-558 
Single-unit recording, 98-100 
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relevance to perception, 100 
Sinusoidal tone, 198,198f 
Sinusoidal waves, 270 
Situational-focus groups, 434-435 
Situation-focused regulation, 436f 
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auditory awareness, 233 
brain activity, 302t 
EEG patterns, 244-247 
need for timely, 243-244 
see also Deep sleep 
Sleep deprivation, 243-244 
Sleep stages, 280f 
Sleepwalking, 243 
Slow Oscillations (SO), 282-286 
Slow Wave Sleep (SWS), 243f, 247-248 
Small world networks, 249, 250f, 293f 
Smile control, 52 
Social behavior and emotion, 433 
Social cognition, 445 
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like- and unlike-others, 460-462 
organizational framework, 449-450 
perspective, 446-447 
Social faux pas , 461 
Social inference, 461 
Social maturity, 406-407 
age of maturity, 406-407 
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Society model of brain, 288 
Software improvements in MRIs, 
582-583 

Somatosensory homunculus, 144f 
Somatosensory pathways, 71 f, 72f 
Somatosensory regions, 143 
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interaural level difference, 210-211 
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Sound processing, 196-197 
limited and large capacity, 197 
orders of magnitude and levels of 
analysis, 197 
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sound perception, 196-197 
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Sound recognition, 215f 
Sound shadow, 210-211 
Sounds 

auditory processing, 200-202 
detecting/discriminating, 207-208 
intensity scale, 199 
localizing, 210-213 
physical features of, 198-199 
psychological aspects, 199-200 
Source memory, 325-326 
Span of immediate memory, 309 
Spatial attention, 212-213 
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Spectrogram, 198f, 219-220 
Spectrograph, 219-220 
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decoding, 213-215, 220, 222 
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pathways, 384f 

planning and producing, 383-384 
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Speech perception, 218-226 

damage to speech perceptual areas, 
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formants, 219-220 
functional mapping, 222-223 
history of research, 218-220 
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motor theory, 220-221 
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phonemes, 218, 221 
reconstructing language pathways, 
225 
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units of analysis, 221-222 
working model, 226 
Speech production, 223 
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Spikes, 65,526,527 
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Spiking patterns, 76-79 
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Spontaneous activity and EEGs, 559 
Spontaneous attention, 53-54 
Spritzers, 6,151-152 
Squire, Larry R., 55f 

recent study of human memory, 
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SSRI (selective serotonin reuptake 
inhibitors), 511, 533 
Statistical regularities, 481 
Stem cells, 471 f 
Step pyramid, 76f 
Stereoscopic depth, 159 
Stimulus-driven activity, 182 
Stimulus-driven attention, 276 
Strabismus (lazy eye), 265-266 
Stress hormones, 432f 
Striate cortex, 164-166 
Stroop Color-naming Task, 51-52, 51f 
emotional, 432-433 
executive attention, 273 
frontal lobe dysfunction, 415 
hypnosis in, 296 
Structural connectivity, 82 
Study methods (in this course), 30 
Stuss, Donald, 402 
Subcortex, 151-153 
Subcortical organs, 151 
Subcortical pathways, 177 
Subliminal learning, 330t 
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Subtraction method, 117-118,118f, 
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469-470 

Superior temporal gyrus, 376f 
Superior temporal sulcus (STS), 171, 
488 
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Switching gears, 405 
Syllables, 197, 381-382 
Sylvian fissure, 143,147-148, 

375, 403 

Symbol systems, 360 
Symbolic interpretation, 29 
Symbolic processing, 87-88 
Synapses, 6, 66f, 67f 
basics, 525f 

cell adhesion, 544, 546f 
chemical self-regulation, 525-526 
classical vs. nonclassical, 524 
glutamate, 543-548 
on-off switches, 524 
traffic control points, 533 
Synaptic connections, 516 
Synaptic loss, 478 


Synaptic transmission, 528-539 
cleaning up synapse, 538-539 
detect signaling, 534-535 
GABA, 528-531 
glutamate, 528-529 
ion channels opened by other 
molecules, 535-538 
neuroglia, 531 
neuromodulators, 528 
neurotransmitters, 528 
production, release, disposal of 
neurotransmitters, 531-532 
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548f 

synaptic release, 532 
traffic control points, 533 
Synaptogenesis, 477, 516-517 
Synchrony, 252-254, 261 
compared to AM/FM radio, 262 
for gain control, 254-255 
gamma, 264-267 
Synesthesia, 141-142 
Synonyms, 389 
Syntactic ambiguity, 373f 
Syntactic analysis, 372 
Syntactic knowledge, 392 
Syntactic processes, 494f 
Syntax, 390-391, 392 
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(SAM), 566-567 

System consolidation, 323, 324-325 

Talairach coordinates, 131 
Task switching, 353f 
Taste, 148 

Temporal codes, 76-79 
Temporal coding, 76-77, 250 
Temporal complexity, 292 
Temporal envelope, 222 
Temporal lobe, 13, 129-130, 129f, 

146,167 
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junction, 387 

Ten orders of magnitude and time, 
6-7 

Teratogens, 474 
Texture perception, 159 
Thalamo-cortical system, 81,127, 249 
Thalamus, 44,135,136f 
interconnectivity with cortex, 
149-150, 245, 247-250 
problem-solving, 353-354 
sensory pathways, 72f 
terminology, 13 

Theory of mind (TOM), 446, 447, 450 
higher-order abilities, 457-460 
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Theory of mind model (Baron-Cohen), 
450f 

Theory of mind module (TOMM), 447, 
450,450f 

Theory theories, 447 
Theta waves/rhythms, 89-91, 

105-106,107t, 109f 
alpha/theta feedback, 297-298 
in deep sleep, 286 
memory retrieval, 340 
multiple roles of, 268-269 
theta-gamma multiplexing, 268f 
Thinking, implicit, 362-363 
Thinking working memory, 45 
Third person perspective, 446-447 
Threat Simulation theory of dreams, 
280 

Threshold, 190,199 
Timbre, 199 
Time 

reaction, 7t, 45 
sound processing, 197 
ten orders of magnitude, 6-7 
Time-frequency graphs, 78, 79f 
Time-limited active memory, 347 
Tonal languages, 386 
Tonographic organization, 70 
Tononi, Giulio, 242 
Tonotopy, 201, 208 
Top-down attention, 407-408 
Top-down processing, 76, 81 
Top-down relationships, 427 
Topographic organization, 70 
Topographical maps, 109f, 251f 
Touch response, 447-448 
Tourette's syndrome, 416 
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350,353-354,352f, 410,411f 
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Tractus solitarius nucleus, 135f 
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plasticity due to learning, 232f 
problem solving, 357 
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Transcranial magnetic stimulation 
(TMS), 108-113,188-189 
cortical feedback, 188-189 
phosphenes and VI stimulation, 188 
Transcription, 511, 519 
Transcription factors, 520 
Transformation, 402 
Transient memory consolidation, 58f 
Transitional probability (TP), 481, 482 
Transitions, vocal, 381 
Translation, 511, 519 
Transverse relaxation time, 572 


Traumatic brain injury (TBI), 416 
Trigeminal main sensory nucleus, 135f 
Trigeminal mesencephalic nucleus, 
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Trigeminal motor nucleus, 135f 
Trigeminal spinal nucleus, 135f 
Triune brain, 421-423 
Trochlear nucleus, 135f 
Troxler, Ignaz Paul Vital, 174-175 
True negative, 185 
Tryptophan, 510 

Tuned, broadly/narrowly, 207-208 
Twin studies, 553t 

Umami, 528, 535, 536f 
Unconditioned stimuli (UCS), 437 
Unconscious brain events, 121-122 
contemporary study of, 27-28 
Unconscious perception, 28f, 189-191 
U nconsciousness 

19th century research, 20-25 
mediated tasks, 242-255, 257t 
Unit modules, 44 
Unpredictability, 41 
Unusual states, 295-301 
Up-regulation, 434-435 
Utilization behavior, 414^15 

Vasopressin/oxytocin, 540t 
Ventral anterior nucleus, 150f 
Ventral brainstem, 417 
Ventral lateral nucleus, 150f 
Ventral object areas, damage to, 
178-181 

Ventral pallidum, 438 
Ventral pathway, 167-169 
Ventral posterolateral nucleus, 150f 
Ventral posteromedial nucleus, 150f 
Ventral visual pathway, 166 
Ventral visual stream, 14 
Ventricles, 136 

Ventricular proliferative zone, 469-470 
Ventrolateral prefrontal cortex 

(VLPFC), 318f, 33If, 337,400 
Ventromedial prefrontal cortex 
(VM-PFC), 457-458,460 
Verbal dyspraxia, 516 
Verbal rehearsal, 35-36 
Veridical decision-making, 411 
Very large brain capacities, 43^4 
Very large-capacity functions, 43t 
Vesalius, Andreas, 12f, 13 
Vesicles, 524, 528-529, 533f 
Viagra, 6f 

Visceral information, 426 
Vision, 8,157 


and brain activity, 182-187 
binocular rivalry, 182-185, 

265-266 

constructive perception, 185-187 
multistable perception, 182 
neural correlates of object 
recognition, 187 
visual detection, 185 
purpose of, 158-159 
unconscious vs. conscious, 28f 
Visual agnosia, 178-181, 365-366 
Visual ambiguity, 80 
Visual areas, damage to, 176-177 
brain areas and visual defects, 178f 
dorsal parietal areas, 181-182 
early visual areas, 176 
ventral object areas, 178-181 
Visual awareness 

lesion studies, 176-182 
cortical color blindness, 177-178 
damage to dorsal parietal areas, 
181-182 

damage to early visual areas, 176 
damage to ventral object areas, 
178-181 

extrastriate lesions, 177-178 
motion blindness, 177 
VI and blindsight, 176-177 
visual agnosia, 178-181 
manipulation of, 187-191 
TMS, 188-189 

unconscious perception, 189-191 
unconscious perception, 189-191 
Visual consciousness, 170-173 

hierarchical and interactive theories, 
173 

humans perceiving humans, 

171-172 

Visual cortex (VI), 164-166 
Visual detection, 185 
Visual fading and eye movements, 
174-175 

Visual features, 164-165 
Visual grouping, 159-160 
Visual imagery, 46-47,48 
Visual maps, 74f 

see also Visuotopic maps 
Visual neglect, 181,192 
Visual object perception, 89 
Visual pathways, 71 f, 72f 
Visual perception, 158-160 
Visual phantoms, 186 
Visual pop-out effect, 53 
Visual processing, 166f 
Visual prototypes, 359-360 
Visual system, 160-170 
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area MT, 166-167 
extrastriate visual areas, 166 
lateral geniculate nucleus, 163-164 
location, 168 

object recognition, 169, 255 
primary visual cortex, 163,164-166 
retina, 160-162 

ventral and dorsal pathways, 167-169 
visual areas of brain, 167f 
Visual working memory, 45-46, 337f 
Visualizing neural communication, 
78-79 

Visuospatial deficit, 504f 
Visuospatial sketchpad, 35-36 
Visuotopic maps, 250, 252, 265f, 274 
Vitamins, 512 
Vocal tract, 381, 382f 
Vocalization, 385, 387f 
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PLATE 1 A first approach to the anatomy of the brain. A. Always keep in mind how the brain is 
situated in the human head. It's the first step in appreciating the spatial layout of the cortex, which 
is filled with some 85% white matter, the shielded "highways' that link all the major regions to each 
other. Bottom right: The cortex is mounted on the brainstem and subcortex, which flows up from the 
spinal cord. The event-related EEG is a reminder that the young lady in A has a constant, dynamic 
flow of massive signal traffic flowing through her brain, which we can pick up with surface EEG. 




Sagittal section 


Coronal section 


PLATE 2 Use the human head (upper left) to stay oriented. We can see the corpus callosum from 
the left side. The brain has three major planes of section to keep in mind. 










PLATE 3 The specialized regions 
of the cortex, the huge covering of 
the brain. Cortex is a flat sheet that is 
folded into the upper cranium. Notice 
the colored regions - the major func¬ 
tions of the cortex. It is the cortex that 
is believed to support the specific 
contents of conscious experiences. Its 
posterior half is sensory, its front half 
is motor and "future directed' - cogni¬ 
tion, working memory, planning, 
decision-making. 



PLATE 4 Neurons pervade the 
body, and the spinal cord is a great 
highway channel between the brain 
and the rest of the body. Left, we see 
the brain from a left medial perspec¬ 
tive. Notice the canonical directions, 
which are like North, South, East 
and West in geography. The simplest 
terms are given on top (Up, Down, 
etc.). But anatomists always use the 
Latin-based words. Use this figure for 
reference if you need to understand a 
brain figure in the book. 













Hippocampus and amygdala in both medial temporal lobes 



Medial temporal lobes 


Bottom view (in yellow and green) 


PLATE 5 Again, begin with the 
human head, and notice how the two 
hippocampi are nestled inside of each 
of the two temporal lobes. A vertical 
slice from ear to ear (coronal) only 
shows the hippocampi as small circu¬ 
lar disks (red). On the bottom, if you 
imagine craning the head backward, 
you can visualize the location of the 
medial temporal lobe (MTL) which 
contains the two hippocampi. Those 
structures are crucial for emotion, 
vision, and memory. 



PLATE 6 Upper Left : The white 
matter tracts emerge in a great' foun¬ 
tain' of fiber tracts from the thalamus. 
Other fiber tracts run in all the major 
directions. Below and to the right are 
tractographs, specialized MRI scans 
that show the massive highways 
between all major parts of the brain. 
Almost all are bi-directional. Source : 
Upper left, Izhikevich & Edelman, 
2008; right side: Hofer & Frahm, 2006. 















Autonomic regulation 



Gianaros, Fig. 3: Stressor- 
evoked activity correlated with 
blood pressure reactivity 
(meta-analysis) 



Wager, part I, Fig. 5, and 
Part II, Fig. 3: Social 
threat responses mediate 
heart rate increases 



Urry, Fig. 3: increases during 
emotion regulation correlated 
with autonomic indicators of 
cognitive demand 



Endocrine and immune regulation 


Ohira, Fig. 3: Stress- 
related correlates of 
plasma NK cell 
increases 

O’Connor, Fig. 1: 
Activity to grief- 
evoking words 
corroelated with IL-1 (3 
increases 



King, Fig. 4: Trauma 
script related activity 
correlated with ACTH 
response 

Eisenberger, Fig. 2: 
Activity during social 
exclusion correlated 
with endotoxin- 
induced IL-6 


Autonomic brainstem activity 



Cechetto, Fig. 5: 
Valsalva 


Wager, Part II, Fig. 5: 
Social threat 


PLATE 7 The body and brain are highly interconnected. Upper left, we see how the medial 
view of the brain (facing right) is well-connected with endocrine system (hormonal) and immune 
regulation, and with autonomic regulation (heart, lungs, stomach, intestines, and blood vessels). 
Humans have no voluntary control over these functions, unlike our external muscles of the body 
and head. Source : Lane & Wager, 2009. 



PLATE 8 A wider perspective. On the upper left, we see a person looking at a rotating visual 
disk, which triggers widespread brain activity, beginning in area VI (dark blue). VI is as big as 
a credit card, but it is folded inside the occipital lobe. Below, we see a comparison to a fruit fly 
brain, which has perhaps 100 000 neurons. A single neuron is shown to the right, followed by a 
single organic molecule, an amino acid (because of the amine fraction (NH), and a carboxyl frac¬ 
tion (COOH). All the vertices in the diagram stand for carbon atoms. R refers to a side chain, which 
could be quite variable. All levels of analysis, from molecules to gross anatomy, are vitally impor¬ 
tant. All proteins and many neurotransmitters involve amino acids. 








































